Skip to main content

Module iceberg_pk_index_sink

Module iceberg_pk_index_sink 

Source
Expand description

Manager for the Iceberg pk-index sink path. Owns per-sink commit coordinators that drive iceberg commit_epoch ahead of hummock commit_epoch and persist exactly-once state via pending_sink_state.

This is intentionally separate from crate::manager::sink_coordination (which serves V1/V2 sinks via gRPC). Future responsibilities such as per-sink compaction will live alongside the per-sink commit coordinator here.

Modules§

backfill
coordinator 🔒
Per-sink iceberg pk-index commit coordinator. This is a plain struct (no background task / mpsc): the crate::manager::iceberg_pk_index_sink::IcebergPkIndexSinkManager holds one per registered sink behind a per-sink async mutex and calls IcebergPkIndexSinkCoordinator::pre_commit / IcebergPkIndexSinkCoordinator::commit directly from the barrier-completion path. The barrier path already serializes pre-commit/commit per epoch, so the coordinator never needs its own request queue.
manager 🔒

Structs§

IcebergPkIndexSinkManager
Manager for the Iceberg pk-index sink path, cheap to clone.

Functions§

build_iceberg_config
Build an IcebergConfig from a PbSink, filling secret refs along the way. Used at CREATE SINK time and during recovery to (re-)register the commit coordinator.
is_iceberg_pk_index_sink
Returns true if the given sink properties identify a Iceberg pk-index sink (i.e. an iceberg sink with enable_pk_index = 'true').