Expand description
This rule materializes a LogicalIcebergIntermediateScan to the final
LogicalIcebergScan with delete file anti-joins.
This is the final step in the Iceberg scan optimization pipeline:
LogicalSource->LogicalIcebergIntermediateScan- Predicate pushdown and column pruning on
LogicalIcebergIntermediateScan LogicalIcebergIntermediateScan->LogicalIcebergScan(this rule)
At this point, the intermediate scan has accumulated:
- The predicate to be pushed down to Iceberg
- The output column indices for projection
This rule:
- Reads file scan tasks from Iceberg (data files and delete files)
- Creates the
LogicalIcebergScanfor data files with pre-computed splits - Creates anti-joins for equality delete and position delete files
- Adds a project if output columns differ from scan columns
Structsยง
Functionsยง
- build_
column_ ๐catalogs - Builds a mapping of column names to their catalogs by looking them up from a catalog map.
- build_
equal_ ๐conditions - Builds equality conditions between two sets of input references.
- build_
equality_ delete_ hashjoin_ scan - build_
position_ delete_ hashjoin_ scan - empty_
table_ ๐plan - Returns an empty table plan with the same schema as the scan.
- set_
project_ ๐field_ ids - Sets the project field IDs for a list of files based on column names.