Struct Scheduler 

pub(super) struct Scheduler {
    default_hash_mapping: ActorAlignmentMapping,
    default_singleton_worker: WorkerId,
    dynamic_mapping_fn: Box<dyn Fn(usize, Option<usize>) -> Result<ActorAlignmentMapping>>,
}

Scheduler schedules the distribution of fragments in a stream graph.

Fields

default_hash_mapping: ActorAlignmentMapping

The default hash mapping for hash-distributed fragments, if there’s no requirement derived.

default_singleton_worker: WorkerId

The default worker for singleton fragments, if there’s no requirement derived.

dynamic_mapping_fn: Box<dyn Fn(usize, Option<usize>) -> Result<ActorAlignmentMapping>>

Use to generate mapping if a vnode count other than the default is required.

Implementations

impl Scheduler

pub fn new( streaming_job_id: u32, workers: &HashMap<u32, WorkerNode>, default_parallelism: NonZeroUsize, expected_vnode_count: usize, ) -> MetaResult<Self>

Create a new Scheduler with the given workers and the default parallelism.

Each hash-distributed fragment will be scheduled to at most default_parallelism parallel units, in a round-robin fashion on all compute nodes. If the default_parallelism is None, all workers will be used.

For different streaming jobs, we even out possible scheduling skew by using the streaming job id as the salt for the scheduling algorithm.

pub fn schedule( &self, graph: &CompleteStreamFragmentGraph, ) -> MetaResult<HashMap<GlobalId<{ IdCategory::Fragment }>, Distribution>>

Schedule the given complete graph and returns the distribution of each building fragment.