Struct StreamTableScan

pub struct StreamTableScan {
    pub base: PlanBase<Stream>,
    core: TableScan,
    batch_plan_id: PlanNodeId,
    stream_scan_type: StreamScanType,
}

StreamTableScan is a virtual plan node to represent a stream table scan. It will be converted to stream scan + merge node (for upstream materialize) + batch table scan when converting to MView creation request.

Fields

base: PlanBase<Stream>

core: TableScan

batch_plan_id: PlanNodeId

stream_scan_type: StreamScanType

Implementations

impl StreamTableScan

pub fn new_with_stream_scan_type( core: TableScan, stream_scan_type: StreamScanType, ) -> Self

pub fn table_name(&self) -> &str

pub fn core(&self) -> &TableScan

pub fn to_index_scan( &self, index_name: &str, index_table_catalog: Arc<TableCatalog>, primary_to_secondary_mapping: &BTreeMap<usize, usize>, function_mapping: &HashMap<FunctionCall, usize>, stream_scan_type: StreamScanType, ) -> StreamTableScan

pub fn stream_scan_type(&self) -> StreamScanType

fn get_upstream_state_table(&self) -> &TableCatalog

pub fn build_backfill_state_catalog( &self, state: &mut BuildFragmentGraphState, ) -> TableCatalog

Build catalog for backfill state

Schema: | vnode | pk … | backfill_finished | row_count |

key: | vnode | value: | pk … | backfill_finished | row_count |

When we update the backfill progress, we update it for all vnodes.

pk refers to the upstream pk which we use to track the backfill progress.

vnode is the corresponding vnode of the upstream’s distribution key. It should also match the vnode of the backfill executor.

backfill_finished is a boolean which just indicates if backfill is done.

row_count is a count of rows which indicates the # of rows per executor. We used to track this in memory. But for backfill persistence we have to also persist it.

FIXME(kwannoel):

• Across all vnodes, the values are the same.
• e.g. | vnode | pk … | backfill_finished | row_count | | 1002 | Int64(1) | t | 10 | | 1003 | Int64(1) | t | 10 | | 1003 | Int64(1) | t | 10 |

Eventually we should track progress per vnode, to support scaling with both mview and the corresponding no_shuffle_backfill. However this is not high priority, since we are working on supporting arrangement backfill, which already has this capability.

impl StreamTableScan

pub fn adhoc_to_stream_prost( &self, state: &mut BuildFragmentGraphState, ) -> Result<PbStreamNode, SchedulerError>

Methods from Deref<Target = PlanBase<Stream>>

pub fn clone_with_new_plan_id(&self) -> Self

pub fn clone_with_new_distribution(&self, dist: Distribution) -> Self

Clone the plan node with a new distribution.

Panics if the plan node is not physical.

Trait Implementations

impl Clone for StreamTableScan

fn clone(&self) -> StreamTableScan

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl ColPrunable for StreamTableScan

fn prune_col( &self, _required_cols: &[usize], _ctx: &mut ColumnPruningContext, ) -> PlanRef

Transform the plan node to only output the required columns ordered by index number.

impl Debug for StreamTableScan

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Deref for StreamTableScan

type Target = PlanBase<Stream>

The resulting type after dereferencing.

fn deref(&self) -> &Self::Target

Dereferences the value.

impl Distill for StreamTableScan

fn distill<'a>(&self) -> XmlNode<'a>

fn distill_to_string(&self) -> String

impl ExprRewritable for StreamTableScan

fn has_rewritable_expr(&self) -> bool

fn rewrite_exprs(&self, r: &mut dyn ExprRewriter) -> PlanRef

impl ExprVisitable for StreamTableScan

fn visit_exprs(&self, v: &mut dyn ExprVisitor)

impl Hash for StreamTableScan

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for StreamTableScan

fn eq(&self, other: &StreamTableScan) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl PlanNodeMeta for StreamTableScan

const NODE_TYPE: PlanNodeType = PlanNodeType::StreamTableScan

type Convention = Stream

fn plan_base(&self) -> &PlanBase<Stream>

Get the reference to the PlanBase with corresponding convention.

fn plan_base_ref(&self) -> PlanBaseRef<'_>

Get the reference to the PlanBase with erased convention.

impl PlanTreeNode for StreamTableScan

fn clone_with_inputs(&self, inputs: &[PlanRef]) -> PlanRef

Clone the node with a list of new inputs.

fn inputs(&self) -> SmallVec<[PlanRef; 2]>

Get input nodes of the plan.

impl PredicatePushdown for StreamTableScan

fn predicate_pushdown( &self, _predicate: Condition, _ctx: &mut PredicatePushdownContext, ) -> PlanRef

Push predicate down for every logical plan node.

impl StreamNode for StreamTableScan

fn to_stream_prost_body( &self, _state: &mut BuildFragmentGraphState, ) -> PbNodeBody

impl ToBatch for StreamTableScan

fn to_batch(&self) -> Result<PlanRef, RwError>

to_batch is equivalent to to_batch_with_order_required(&Order::any())

fn to_batch_with_order_required( &self, required_order: &Order, ) -> Result<PlanRef, RwError>

convert the plan to batch physical plan and satisfy the required Order

impl ToDistributedBatch for StreamTableScan

fn to_distributed(&self) -> Result<PlanRef, RwError>

to_distributed is equivalent to to_distributed_with_required(&Order::any(), &RequiredDist::Any)

fn to_distributed_with_required( &self, required_order: &Order, required_dist: &RequiredDist, ) -> Result<PlanRef, RwError>

insert the exchange in batch physical plan to satisfy the required Distribution and Order.

impl ToLocalBatch for StreamTableScan

fn to_local(&self) -> Result<PlanRef, RwError>

fn to_local_with_order_required( &self, required_order: &Order, ) -> Result<PlanRef, RwError>

Convert the plan to batch local physical plan and satisfy the required Order

impl ToStream for StreamTableScan

fn to_stream(&self, _ctx: &mut ToStreamContext) -> Result<PlanRef, RwError>

to_stream is equivalent to to_stream_with_dist_required(RequiredDist::Any)

fn logical_rewrite_for_stream( &self, _ctx: &mut RewriteStreamContext, ) -> Result<(PlanRef, ColIndexMapping), RwError>

logical_rewrite_for_stream will rewrite the logical node, and return (new_plan_node, col_mapping), the col_mapping is for original columns have been changed into some other position.

fn to_stream_with_dist_required( &self, required_dist: &RequiredDist, ctx: &mut ToStreamContext, ) -> Result<PlanRef, RwError>

convert the plan to streaming physical plan and satisfy the required distribution

impl TryToBatchPb for StreamTableScan

fn try_to_batch_prost_body(&self) -> Result<NodeBody, SchedulerError>

impl Eq for StreamTableScan

impl PlanNode for StreamTableScan

impl StructuralPartialEq for StreamTableScan

impl ToPb for StreamTableScan