Module iceberg 

Modules

commit 🔒

commit_retry

Shared iceberg commit-with-retry primitive used by both the V1/V2 sink committer and the V3 sink coordinator worker. Wraps the standard pattern of “reload the table, build an action against the freshly-loaded snapshot, commit, retry on transient errors only”.

config 🔒

create_table 🔒

engine_options 🔒

metadata 🔒

position_delete 🔒

Shared iceberg position-delete (Puffin deletion vector) helpers.

prometheus 🔒

writer 🔒

Structs

IcebergCommitResult

IcebergConfig

IcebergOrderKeyField

IcebergPositionDeleteCommitResult

IcebergSink

IcebergSinkCommitter

IcebergSinkWriterArgs

IcebergSinkWriterInner

IcebergWriterMetrics

Enums

CompactionType

Compaction type for Iceberg sink

IcebergMetadataTableType

The supported per-table Iceberg metadata relations.

IcebergSinkWriter

IcebergWriteMode

Constants

COMPACTION_DELETE_FILES_COUNT_THRESHOLD

COMPACTION_INTERVAL_SEC

COMPACTION_MAX_SNAPSHOTS_NUM

COMPACTION_SMALL_FILES_THRESHOLD_MB

COMPACTION_TARGET_FILE_SIZE_MB

COMPACTION_TRIGGER_SNAPSHOT_COUNT

COMPACTION_TYPE

COMPACTION_WRITE_PARQUET_COMPRESSION

COMPACTION_WRITE_PARQUET_MAX_ROW_GROUP_BYTES

COMPACTION_WRITE_PARQUET_MAX_ROW_GROUP_ROWS

DEFAULT_COMPACTION_MAX_SNAPSHOTS_NUM

ENABLE_COMPACTION

ENABLE_MANIFEST_REWRITE

ENABLE_PK_INDEX

ENABLE_SNAPSHOT_EXPIRATION

FORMAT_VERSION

ICEBERG_COMPACTION_TYPE_AUTO

ICEBERG_COMPACTION_TYPE_FILES_WITH_DELETE

ICEBERG_COMPACTION_TYPE_FULL

ICEBERG_COMPACTION_TYPE_SMALL_FILES

ICEBERG_COW_BRANCH

ICEBERG_DEFAULT_WRITE_PARQUET_MAX_ROW_GROUP_BYTES

ICEBERG_ENGINE_OPTION_KEYS

ICEBERG_SINK

ICEBERG_WRITE_MODE_COPY_ON_WRITE

ICEBERG_WRITE_MODE_MERGE_ON_READ

MANIFEST_REWRITE_MIN_COUNT_TO_MERGE

MANIFEST_REWRITE_TARGET_SIZE_BYTES

ORDER_KEY

PARQUET_CREATED_BY

PARTITION_DATA_ID_START

SNAPSHOT_EXPIRATION_CLEAR_EXPIRED_FILES

SNAPSHOT_EXPIRATION_CLEAR_EXPIRED_META_DATA

SNAPSHOT_EXPIRATION_MAX_AGE_MILLIS

SNAPSHOT_EXPIRATION_RETAIN_LAST

WRITE_MODE

Functions

commit_branch

create_and_validate_table_impl

is_iceberg_engine_option

parse_order_key_exprs

parse_parquet_compression

Parse compression codec string to Parquet Compression enum

parse_partition_by_exprs

read_dv_positions_from_data_file

Reads the deletion-vector positions of a single Puffin DV DataFile.

read_parquet_position_deletes_from_file

Reads the positions stored in a V2 Parquet position-delete file into a [DeleteVector].

read_position_deletes_from_file

Reads the deleted positions of a single position-delete DataFile regardless of on-disk format,

resolve_partition_spec

Resolves a partition spec by id from the table metadata, with a consolidated error message. Shared spec-lookup mechanic for the pk-index merger, commit coordinator, and sink commit paths.

resolve_partition_type

Resolves the partition [StructType] for the given spec_id against schema.

scan_iceberg_metadata

Read one Iceberg metadata relation and emit bounded DataChunks.

serialize_data_files_default_spec

Serializes files against the table’s current default partition spec, after truncating oversized column statistics. Shared by the iceberg sink writer’s commit-metadata generation and the pk-index merger’s delete-file serialization.

should_enable_iceberg_cow

truncate_datafile

Truncate large column statistics from DataFile BEFORE serialization.

try_matches_arrow_schema

Try to match our schema with iceberg schema.

validate_order_key_columns

write_dv_puffin_file

Writes delete_vector as a single Puffin deletion-vector blob referencing data_file_path, and returns its [DataFile] metadata (content PositionDeletes, format Puffin) with referenced_data_file set.

write_parquet_position_delete_file

Writes delete_vector as a single file-scoped Parquet position-delete file referencing data_file_path, and returns its [DataFile] metadata (content PositionDeletes, format Parquet) with referenced_data_file set.

Type Aliases

PositionDeleteFileWriterBuilderType

The concrete V2 position-delete Parquet writer builder type, shared by the iceberg sink writer and the pk-index position-delete merger.