Struct ActorGraphBuilder

pub struct ActorGraphBuilder {
    distributions: HashMap<GlobalId<{ IdCategory::Fragment }>, Distribution>,
    existing_distributions: HashMap<GlobalId<{ IdCategory::Fragment }>, Distribution>,
    fragment_graph: CompleteStreamFragmentGraph,
    cluster_info: StreamingClusterInfo,
}

ActorGraphBuilder builds the actor graph for the given complete fragment graph, based on the current cluster info and the required parallelism.

Fields

distributions: HashMap<GlobalId<{ IdCategory::Fragment }>, Distribution>

The pre-scheduled distribution for each building fragment.

existing_distributions: HashMap<GlobalId<{ IdCategory::Fragment }>, Distribution>

The actual distribution for each existing fragment.

fragment_graph: CompleteStreamFragmentGraph

The complete fragment graph.

cluster_info: StreamingClusterInfo

The cluster info for creating a streaming job.

Implementations

impl ActorGraphBuilder

pub fn new( streaming_job_id: u32, fragment_graph: CompleteStreamFragmentGraph, cluster_info: StreamingClusterInfo, default_parallelism: NonZeroUsize, ) -> MetaResult<Self>

Create a new actor graph builder with the given “complete” graph. Returns an error if the graph is failed to be scheduled.

fn get_distribution( &self, fragment_id: GlobalId<{ IdCategory::Fragment }>, ) -> &Distribution

Get the distribution of the given fragment. Will look up the distribution map of both the building and existing fragments.

fn build_locations( &self, actor_locations: BTreeMap<GlobalId<{ IdCategory::Actor }>, WorkerSlotId>, ) -> Locations

Convert the actor location map to the Locations struct.

pub fn generate_graph( self, env: &MetaSrvEnv, job: &StreamingJob, expr_context: ExprContext, ) -> MetaResult<ActorGraphBuildResult>

Build a stream graph by duplicating each fragment as parallel actors. Returns ActorGraphBuildResult that will be further used to build actors on the compute nodes.

fn build_actor_graph( &self, id_gen: GlobalIdGen<{ IdCategory::Actor }>, ) -> MetaResult<ActorGraphBuildStateInner>

Build actor graph for each fragment, using topological order.

fn build_actor_graph_fragment( &self, fragment_id: GlobalId<{ IdCategory::Fragment }>, state: &mut ActorGraphBuildState, ) -> MetaResult<()>

Build actor graph for a specific fragment.