Expand description
Defines all kinds of node in the plan tree, each node represent a relational expression.
We use a immutable style tree structure, every Node are immutable and cannot be modified after
it has been created. If you want to modify the node, such as rewriting the expression in a
ProjectNode
or changing a node’s input node, you need to create a new node. We use Rc as the
node’s reference, and a node just storage its inputs’ reference, so change a node just need
create one new node but not the entire sub-tree.
So when you want to add a new node, make sure:
- each field in the node struct are private
- recommend to implement the construction of Node in a unified
new()
function, if have multi methods to construct, make they have a consistent behavior - all field should be valued in construction, so the properties’ derivation should be finished
in the
new()
function.
Re-exports§
pub use generic::PlanAggCall;
pub use generic::PlanAggCallDisplay;
Modules§
- convert 🔒
- derive 🔒
- This module contains the generic plan nodes that are shared by all the plan nodes. They are meant to reuse the common fields between logical, batch and stream nodes.
- to_
prost 🔒
Macros§
- impl plan node downcast fn for each node.
- impl
PlanNodeType
fn for each node.
Structs§
- The marker for batch convention.
BatchDelete
implementssuper::LogicalDelete
BatchExchange
imposes a particular distribution on its input without changing its content.- Extra fields for batch plan nodes.
BatchFilter
implementssuper::LogicalFilter
BatchGroupTopN
implementssuper::LogicalTopN
to find the top N elements with a heapBatchHashJoin
implementssuper::LogicalJoin
with hash table. It builds a hash table from inner (right-side) relation and then probes with data from outer (left-side) relation to get output rows.BatchHopWindow
implementssuper::LogicalHopWindow
to evaluate specified expressions on input rowsBatchInsert
implementssuper::LogicalInsert
BatchLimit
implementssuper::LogicalLimit
to fetch specified rows from inputBatchMaxOneRow
fetches up to one row from the input, returning an error if the input contains more than one row at runtime.BatchNestedLoopJoin
implementssuper::LogicalJoin
by checking the join condition against all pairs of rows from inner & outer side within 2 layers of loops.BatchProject
implementssuper::LogicalProject
to evaluate specified expressions on input rowsBatchSeqScan
implementssuper::LogicalScan
to scan from a row-oriented tableBatchSort
buffers all data from input and sort these rows by specified order, providing the collation required by user or parent plan node.BatchSource
represents a table/connector source at the very beginning of the graph.BatchSysSeqScan
implementssuper::LogicalSysScan
to scan from a row-oriented tableBatchTopN
implementssuper::LogicalTopN
to find the top N elements with a heapBatchUnion
implementssuper::LogicalUnion
BatchUpdate
implementssuper::LogicalUpdate
- The join predicate used in optimizer
- The marker for logical convention.
LogicalAgg
groups input data by their group key and computes aggregation functions.LogicalApply
represents a correlated join, where the right side may refer to columns from the left side.LogicalCdcScan
reads rows of a table from an external upstream databaseLogicalDedup
deduplicates data on specific columns. It is now used inDISTINCT ON
without anORDER BY
.LogicalDelete
iterates on input relation and delete the data from specified table.LogicalExcept
returns the rows of its first input except any matching rows from its other inputs.LogicalExpand
expands one row multiple times according tocolumn_subsets
and also keeps original columns of input. It can be used to implement distinct aggregation and group set.LogicalFilter
iterates over its input and returns elements for whichpredicate
evaluates to true, filtering out the others.LogicalHopWindow
implements Hop Table Function.LogicalIcebergScan
is only used by batch queries. At the beginning of the batch query optimization,LogicalSource
with a iceberg property would be converted into aLogicalIcebergScan
.LogicalInsert
iterates on input relation and insert the data into specified table.LogicalIntersect
returns the intersect of the rows of its inputs. Ifall
is false, it needs to eliminate duplicates.LogicalJoin
combines two relations according to some condition.LogicalKafkaScan
is only used by batch queries. At the beginning of the batch query optimization,LogicalSource
with a kafka property would be converted into aLogicalKafkaScan
.LogicalLimit
fetches up tolimit
rows fromoffset
LogicalMaxOneRow
fetches up to one row from the input, returning an error if the input contains more than one row at runtime. Only available in batch mode.LogicalMultiJoin
combines two or more relations according to some condition.LogicalOverWindow
performsOVER
window functions to its input.LogicalProject
computes a set of expressions from its input relation.LogicalProjectSet
projects one row multiple times according toselect_list
.LogicalRecursiveUnion
returns the union of the rows of its inputs. note: ifall
is false, it needs to eliminate duplicates.LogicalScan
returns contents of a table or other equivalent objectLogicalShare
operator is used to represent reusing of existing operators. It is the key operator for DAG plan. It could have multiple parents which makes it different from other operators. Currently, it has been used to the following scenarios:LogicalSource
returns contents of a table or other equivalent objectLogicalSysScan
returns contents of a table or other equivalent objectLogicalTableFunction
is a scalar/table function used as a relation (in theFROM
clause).LogicalTopN
sorts the input data and fetches up tolimit
rows fromoffset
LogicalUnion
returns the union of the rows of its inputs. Ifall
is false, it needs to eliminate duplicates.LogicalUpdate
iterates on input relation, set some columns, and inject update records into specified table.LogicalValues
builds rows according to a list of expressions- No extra fields for logical plan nodes.
- The common fields of all plan nodes with different conventions.
- The marker for stream convention.
StreamAsOfJoin
implementssuper::LogicalJoin
with hash tables.StreamCdcTableScan
is a virtual plan node to represent a stream cdc table scan. It will be converted to cdc backfill + merge node (for upstream source)StreamDeltaJoin
implementssuper::LogicalJoin
with delta join. It requires its two inputs to be indexes.StreamExchange
imposes a particular distribution on its input without changing its content.- Extra fields for stream plan nodes.
StreamFilter
implementssuper::LogicalFilter
StreamHashJoin
implementssuper::LogicalJoin
with hash table. It builds a hash table from inner (right-side) relation and probes with data from outer (left-side) relation to get output rows.StreamHopWindow
represents a hop window table function.- Materializes a stream.
StreamProject
implementssuper::LogicalProject
to evaluate specified expressions on input rows.StreamRowMerge
is used for merging two streams with the same stream key and distribution. It will buffer the outputs from its input streams until we receive a barrier. On receiving a barrier, it willProject
their outputs according to the providedlhs_mapping
andrhs_mapping
.StreamShare
will be translated into anExchangeNode
based on its distribution finally.StreamSink
represents a table/connector sink at the very end of the graph.StreamSource
represents a table/connector source at the very beginning of the graph.StreamSourceScan
scans from a shared source. It forwards data from the upstreamStreamSource
, and also backfills data from the external source.- Streaming stateless simple agg.
StreamTableScan
is a virtual plan node to represent a stream table scan. It will be converted to stream scan + merge node (for upstream materialize) + batch table scan when converting toMView
creation request.StreamTopN
implementssuper::LogicalTopN
to find the top N elements with a heapStreamUnion
implementssuper::LogicalUnion
StreamValues
implementsLogicalValues.to_stream()
Enums§
- Why we need
PartitionComputeInfo
? - Reference to
PlanBase
with erased conventions. - each enum value represent a PlanNode struct type, help us to dispatch and downcast
Constants§
- Recursion depth threshold for plan node visitor to send notice to user.
- Notice message for plan node visitor to send to user when the depth threshold is reached.
Traits§
- The trait for column pruning, only logical plan node will use it, though all plan node impl it.
- A marker trait for different conventions, used for enforcing type safety.
- A more sophisticated
Endo
taking into account of the DAG structure ofPlanRef
. In addition toEndo
, one have to specify thecached
function to persist transformedLogicalShare
and their results, and thedag_apply
function will take care to only transform everyLogicalShare
nodes once. - Rewrites expressions in a
PlanRef
. Due toShare
operator, theExprRewriter
needs to be idempotent i.e., applying it more than once to the sameExprImpl
will be a noop on subsequent applications.rewrite_exprs
should only return a plan with the given node modified. To rewrite recursively, callrewrite_exprs_recursive
onRewriteExprsRecursive
. - The common trait over all plan nodes. Used by optimizer framework which will treat all node as
dyn PlanNode
- The trait for accessing the meta data and
PlanBase
for plan nodes. - The trait
PlanNode
really need about tree structure and used by optimizer framework. every plan node should impl it. - See
PlanTreeNode
- See
PlanTreeNode
- See
PlanTreeNode
- The trait for predicate pushdown, only logical plan node will use it, though all plan node impl it.
ToBatch
allows to convert a logical plan node to batch physical node with an optional required order.ToDistributedBatch
allows to convert a batch physical plan to distributed batch plan, by insert exchange node, with an optional required order and distributed.- Converts a batch physical plan to local plan for local execution.
ToStream
converts a logical plan node to streaming physical node with an optional required distribution.- A more sophisticated
Visit
taking into account of the DAG structure ofPlanRef
. In addition toVisit
, one have to specifyvisited
to store and report visitedLogicalShare
nodes, and thedag_visit
function will take care to only visit everyLogicalShare
nodes once. See alsoEndoPlan
.
Functions§
- In order to let expression display id started from 1 for explaining, hidden column names and other places. We will reset expression display id to 0 and clone the whole plan to reset the schema.