Module writer 

Structs

IcebergSinkWriterArgs

IcebergSinkWriterInner

IcebergWriterMetrics

SharedIcebergWriterBuilder 🔒

TaskWriterBuilderWrapper 🔒

Enums

IcebergSinkWriter

IcebergWriterDispatch 🔒

PositionDeleteWriterBuilderType 🔒

PositionDeleteWriterType 🔒

ProjectIdxVec 🔒

None means no projection is needed. Prepare stores the extra partition column index. Done stores the finalized projection indices.

Constants

MAX_COLUMN_STAT_SIZE 🔒

Maximum size for column statistics (min/max values) in bytes. Column statistics larger than this will be truncated to avoid metadata bloat. This is especially important for large fields like JSONB, TEXT, BINARY, etc.

Functions

resolve_equality_delete_field_ids 🔒

resolve_partition_spec

Resolves a partition spec by id from the table metadata, with a consolidated error message. Shared spec-lookup mechanic for the pk-index merger, commit coordinator, and sink commit paths.

resolve_partition_type

Resolves the partition [StructType] for the given spec_id against schema.

serialize_data_files_default_spec

Serializes files against the table’s current default partition spec, after truncating oversized column statistics. Shared by the iceberg sink writer’s commit-metadata generation and the pk-index merger’s delete-file serialization.

truncate_datafile

Truncate large column statistics from DataFile BEFORE serialization.

Type Aliases

DataFileWriterBuilderType 🔒

DeletionVectorWriterBuilderType 🔒

DeletionVectorWriterType 🔒

EqualityDeleteFileWriterBuilderType 🔒

PositionDeleteFileWriterBuilderType

The concrete V2 position-delete Parquet writer builder type, shared by the iceberg sink writer and the pk-index position-delete merger.

PositionDeleteFileWriterType 🔒