Struct PartialGraphManagedBarrierState 

pub(crate) struct PartialGraphManagedBarrierState {
    epoch_barrier_state_map: BTreeMap<u64, BarrierState>,
    prev_barrier_table_ids: Option<(EpochPair, HashSet<TableId>)>,
    pub(crate) create_mview_progress: HashMap<u64, HashMap<ActorId, BackfillState>>,
    pub(crate) list_finished_source_ids: HashMap<u64, HashSet<u32>>,
    pub(crate) load_finished_source_ids: HashMap<u64, HashSet<u32>>,
    pub(crate) cdc_table_backfill_progress: HashMap<u64, HashMap<ActorId, CdcTableBackfillState>>,
    pub(crate) truncate_tables: HashMap<u64, HashSet<u32>>,
    pub(crate) refresh_finished_tables: HashMap<u64, HashSet<u32>>,
    state_store: StateStoreImpl,
    streaming_metrics: Arc<StreamingMetrics>,
}

Part of DatabaseManagedBarrierState

Fields

epoch_barrier_state_map: BTreeMap<u64, BarrierState>

Record barrier state for each epoch of concurrent checkpoints.

The key is prev_epoch, and the first value is curr_epoch

prev_barrier_table_ids: Option<(EpochPair, HashSet<TableId>)>

create_mview_progress: HashMap<u64, HashMap<ActorId, BackfillState>>

Record the progress updates of creating mviews for each epoch of concurrent checkpoints.

The process of progress reporting is as follows:

1. updated by crate::task::barrier_manager::CreateMviewProgressReporter::update
2. converted to ManagedBarrierStateInner in Self::may_have_collected_all
3. handled by Self::pop_barrier_to_complete
4. put in crate::task::barrier_worker::BarrierCompleteResult and reported to meta.

list_finished_source_ids: HashMap<u64, HashSet<u32>>

Record the source list finished reports for each epoch of concurrent checkpoints. Used for refreshable batch source.

load_finished_source_ids: HashMap<u64, HashSet<u32>>

Record the source load finished reports for each epoch of concurrent checkpoints. Used for refreshable batch source.

cdc_table_backfill_progress: HashMap<u64, HashMap<ActorId, CdcTableBackfillState>>

truncate_tables: HashMap<u64, HashSet<u32>>

Record the tables to truncate for each epoch of concurrent checkpoints.

refresh_finished_tables: HashMap<u64, HashSet<u32>>

Record the tables that have finished refresh for each epoch of concurrent checkpoints. Used for materialized view refresh completion reporting.

state_store: StateStoreImpl

streaming_metrics: Arc<StreamingMetrics>

Implementations

impl PartialGraphManagedBarrierState

pub(super) fn new(actor_manager: &StreamActorManager) -> Self

fn new_inner( state_store: StateStoreImpl, streaming_metrics: Arc<StreamingMetrics>, ) -> Self

pub(super) fn is_empty(&self) -> bool

impl PartialGraphManagedBarrierState

fn may_have_collected_all(&mut self) -> Option<Barrier>

This method is called when barrier state is modified in either Issued or Stashed to transform the state to AllCollected and start state store sync when the barrier has been collected from all actors for an Issued barrier.

fn pop_barrier_to_complete(&mut self, prev_epoch: u64) -> BarrierToComplete

impl PartialGraphManagedBarrierState

pub(super) fn collect(&mut self, actor_id: ActorId, epoch: EpochPair)

Collect a barrier from the actor with actor_id.

pub(super) fn transform_to_issued( &mut self, barrier: &Barrier, actor_ids_to_collect: impl IntoIterator<Item = ActorId>, table_ids: HashSet<TableId>, )

When the meta service issues a send_barrier request, call this function to transform to Issued and start to collect or to notify.

Trait Implementations

impl Display for &PartialGraphManagedBarrierState

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.