Expand description
Per-PartialGraphId committed-epoch tracking for the iceberg pk-index sink.
Tracks, per partial graph, the latest checkpoint epoch whose iceberg commit has completed. A
cursor entry exists exactly while a partial graph has a registered pk-index sink:
PartialGraphCommittedEpochs::ensure is called from register_sink and
PartialGraphCommittedEpochs::remove from unregister. Both PartialGraphCommittedEpochs::advance_all
and PartialGraphCommittedEpochs::wait are non-creating, so a partial graph without a sink never
accumulates an entry, and a wait that races an unregister resolves immediately (the caller then
observes the coordinator is gone and errors out) rather than blocking on a resurrected entry that
would never advance.
The key is a PartialGraphId rather than a DatabaseId so that a sink running in an independent
partial graph (e.g. a batch-refresh job, to_partial_graph_id(database_id, Some(job_id))) is tracked
by the very same mechanism: the barrier-completion path advances every completed partial graph, so no
change here is needed to support such jobs. For a normal sink the key is its database’s main graph
(to_partial_graph_id(database_id, None)).
The barrier-completion path advances the cursor on every checkpoint completion (even epochs where
a sink reported nothing), so a merger’s seed wait converges even on idle tables. Waiters block on a
tokio::sync::watch receiver and never hold any other lock.
Note: this is unrelated to RisingWave’s stream watermark abstraction; it is purely a per-partial-graph “iceberg commit has progressed to epoch N” cursor.
Structs§
- Partial
Graph Committed Epochs - Per-
PartialGraphIdmonotonic committed-epoch cursor, backed by onewatchchannel per tracked partial graph.