Struct Source

pub struct Source {
    pub catalog: Option<Rc<SourceCatalog>>,
    pub column_catalog: Vec<ColumnCatalog>,
    pub row_id_index: Option<usize>,
    pub kind: SourceNodeKind,
    pub ctx: OptimizerContextRef,
    pub as_of: Option<AsOf>,
}

Source returns contents of a table or other equivalent object

Fields

catalog: Option<Rc<SourceCatalog>>

If there is an external stream source, catalog will be Some. Otherwise, it is None.

column_catalog: Vec<ColumnCatalog>

row_id_index: Option<usize>

kind: SourceNodeKind

ctx: OptimizerContextRef

as_of: Option<AsOf>

Implementations

impl Source

pub fn is_new_fs_connector(&self) -> bool

pub fn is_iceberg_connector(&self) -> bool

pub fn is_kafka_connector(&self) -> bool

pub fn support_time_travel(&self) -> bool

Currently, only iceberg source supports time travel.

pub fn exclude_generated_columns(self) -> (Self, Option<usize>)

The columns in stream/batch source node indicate the actual columns it will produce, instead of the columns defined in source catalog. The difference is generated columns.

pub fn infer_internal_table_catalog(is_distributed: bool) -> TableCatalog

Source’s state table is partition_id -> offset_info. Its schema is irrelevant to the data’s schema.

Notes on the distribution of the state table (is_distributed)

Source executors are always distributed, but their state tables are special.

StreamSourceExecutor: singleton (only one vnode)

Its states are not sharded by consistent hash.

Each actor accesses (point get) some partitions (a.k.a splits). They are assigned by SourceManager in meta, instead of vnode computed from the partition_id.

StreamFsFetch: distributed by partition_id

Each actor accesses (range scan) splits according to the vnode computed from partition_id. This is a normal distributed table.

Trait Implementations

impl Clone for Source

fn clone(&self) -> Source

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Source

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl GenericPlanNode for Source

fn schema(&self) -> Schema

fn stream_key(&self) -> Option<Vec<usize>>

fn ctx(&self) -> OptimizerContextRef

fn functional_dependency(&self) -> FunctionalDependencySet

impl Hash for Source

where Option<Rc<SourceCatalog>>: Hash, Vec<ColumnCatalog>: Hash, Option<usize>: Hash, SourceNodeKind: Hash, Option<AsOf>: Hash,

fn hash<H: Hasher>(&self, state: &mut H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for Source

where Option<Rc<SourceCatalog>>: PartialEq, Vec<ColumnCatalog>: PartialEq, Option<usize>: PartialEq, SourceNodeKind: PartialEq, Option<AsOf>: PartialEq,

fn eq(&self, other: &Self) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for Source

where Option<Rc<SourceCatalog>>: PartialEq, Vec<ColumnCatalog>: PartialEq, Option<usize>: PartialEq, SourceNodeKind: PartialEq, Option<AsOf>: PartialEq,