Skip to main content

Module iceberg_v3_sink

Module iceberg_v3_sink 

Source
Expand description

Manager for the Iceberg V3 sink path. Owns per-sink commit coordinators that drive iceberg commit_epoch ahead of hummock commit_epoch and persist exactly-once state via pending_sink_state.

This is intentionally separate from crate::manager::sink_coordination (which serves V1/V2 sinks via gRPC). Future V3 responsibilities such as per-sink compaction will live alongside the per-sink commit coordinator here.

Modules§

backfill
coordinator 🔒
Per-sink Iceberg V3 commit coordinator. This is a plain struct (no background task / mpsc): the crate::manager::iceberg_v3_sink::IcebergV3SinkManager holds one per registered sink behind a per-sink async mutex and calls IcebergV3Coordinator::pre_commit / IcebergV3Coordinator::commit directly from the barrier-completion path. The barrier path already serializes pre-commit/commit per epoch, so the coordinator never needs its own request queue.
manager 🔒

Structs§

IcebergV3SinkManager
Manager for the Iceberg V3 sink path, cheap to clone.

Functions§

build_iceberg_config
Build an IcebergConfig from a PbSink, filling secret refs along the way. Used at CREATE SINK time and during recovery to (re-)register the V3 commit coordinator.
is_iceberg_v3_sink
Returns true if the given sink properties identify a Iceberg V3 sink (i.e. an iceberg sink with enable_pk_index = 'true').