Enum Distribution

pub enum Distribution {
    Single,
    SomeShard,
    HashShard(Vec<usize>),
    UpstreamHashShard(Vec<usize>, TableId),
    Broadcast,
}

the distribution property provided by a operator.

Variants

Single

There is only one partition. All records are placed on it.

SomeShard

Records are sharded into partitions, and satisfy the AnyShard but without any guarantee about their placement rules.

HashShard(Vec<usize>)

Records are sharded into partitions based on the hash value of some columns, which means the records with the same hash values must be on the same partition. usize is the index of column used as the distribution key.

UpstreamHashShard(Vec<usize>, TableId)

A special kind of provided distribution which is almost the same as Distribution::HashShard, but may have different vnode mapping.

It exists because the upstream MV can be scaled independently. So we use UpstreamHashShard to force an exchange to be inserted.

Alternatively, Distribution::SomeShard can also be used to insert an exchange, but UpstreamHashShard contains distribution keys, which might be useful in some cases, e.g., two-phase Agg. It also satisfies RequiredDist::ShardByKey.

TableId is used to represent the data distribution(vnode_mapping) of this UpstreamHashShard. The scheduler can fetch TableId’s corresponding vnode_mapping to do shuffle.

Broadcast

Records are available on all downstream shards.

Implementations

impl Distribution

pub fn to_prost( &self, output_count: u32, catalog_reader: &CatalogReader, worker_node_manager: &WorkerNodeSelector, ) -> Result<ExchangeInfo, RwError>

pub fn satisfies(&self, required: &RequiredDist) -> bool

check if the distribution satisfies other required distribution

pub fn dist_column_indices(&self) -> &[usize]

Get distribution column indices. After optimization, only HashShard and Single are valid.

fn get_fragment_id( catalog_reader: &CatalogReader, table_id: &TableId, ) -> Result<u32, RwError>

Trait Implementations

impl Clone for Distribution

fn clone(&self) -> Distribution

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Distribution

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for Distribution

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Hash for Distribution

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for Distribution

fn eq(&self, other: &Distribution) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for Distribution

impl StructuralPartialEq for Distribution