Modules§
- commit 🔒
- commit_
retry - Shared iceberg commit-with-retry primitive used by both the V1/V2 sink committer and the V3 sink coordinator worker. Wraps the standard pattern of “reload the table, build an action against the freshly-loaded snapshot, commit, retry on transient errors only”.
- config 🔒
- create_
table 🔒 - prometheus 🔒
- writer 🔒
Structs§
- Iceberg
Commit Result - Iceberg
Config - Iceberg
Order KeyField - Iceberg
Position Delete Merger Commit Result - Iceberg
Sink - Iceberg
Sink Committer - Iceberg
Sink Writer Args - Iceberg
Sink Writer Inner - Iceberg
Writer Metrics
Enums§
- Compaction
Type - Compaction type for Iceberg sink
- Iceberg
Sink Writer - Iceberg
Write Mode
Constants§
- COMPACTION_
DELETE_ FILES_ COUNT_ THRESHOLD - COMPACTION_
INTERVAL_ SEC - COMPACTION_
MAX_ SNAPSHOTS_ NUM - COMPACTION_
SMALL_ FILES_ THRESHOLD_ MB - COMPACTION_
TARGET_ FILE_ SIZE_ MB - COMPACTION_
TRIGGER_ SNAPSHOT_ COUNT - COMPACTION_
TYPE - COMPACTION_
WRITE_ PARQUET_ COMPRESSION - COMPACTION_
WRITE_ PARQUET_ MAX_ ROW_ GROUP_ BYTES - COMPACTION_
WRITE_ PARQUET_ MAX_ ROW_ GROUP_ ROWS - DEFAULT_
COMPACTION_ MAX_ SNAPSHOTS_ NUM - ENABLE_
COMPACTION - ENABLE_
PK_ INDEX - ENABLE_
SNAPSHOT_ EXPIRATION - FORMAT_
VERSION - ICEBERG_
COMPACTION_ TYPE_ FILES_ WITH_ DELETE - ICEBERG_
COMPACTION_ TYPE_ FULL - ICEBERG_
COMPACTION_ TYPE_ SMALL_ FILES - ICEBERG_
COW_ BRANCH - ICEBERG_
DEFAULT_ WRITE_ PARQUET_ MAX_ ROW_ GROUP_ BYTES - ICEBERG_
SINK - ICEBERG_
WRITE_ MODE_ COPY_ ON_ WRITE - ICEBERG_
WRITE_ MODE_ MERGE_ ON_ READ - ORDER_
KEY - PARQUET_
CREATED_ BY - PARTITION_
DATA_ ID_ START - SNAPSHOT_
EXPIRATION_ CLEAR_ EXPIRED_ FILES - SNAPSHOT_
EXPIRATION_ CLEAR_ EXPIRED_ META_ DATA - SNAPSHOT_
EXPIRATION_ MAX_ AGE_ MILLIS - SNAPSHOT_
EXPIRATION_ RETAIN_ LAST - WRITE_
MODE
Functions§
- commit_
branch - create_
and_ validate_ table_ impl - parse_
order_ key_ exprs - parse_
parquet_ compression - Parse compression codec string to Parquet Compression enum
- parse_
partition_ by_ exprs - resolve_
partition_ spec - Resolves a partition spec by id from the table metadata, with a consolidated error message. Shared spec-lookup mechanic for the pk-index merger, commit coordinator, and sink commit paths.
- resolve_
partition_ type - Resolves the partition [
StructType] for the givenspec_idagainstschema. - serialize_
data_ files_ default_ spec - Serializes
filesagainst the table’s current default partition spec, after truncating oversized column statistics. Shared by the iceberg sink writer’s commit-metadata generation and the pk-index merger’s delete-file serialization. - should_
enable_ iceberg_ cow - truncate_
datafile - Truncate large column statistics from
DataFileBEFORE serialization. - try_
matches_ arrow_ schema - Try to match our schema with iceberg schema.
- validate_
order_ key_ columns
Type Aliases§
- Position
Delete File Writer Builder Type - The concrete V2 position-delete Parquet writer builder type, shared by the iceberg sink writer and the pk-index position-delete merger.