Module source_manager

Structs

• ActorSplitsAssignment 🔒
  Note: the PartialEq and Ord impl just compares the number of splits.
• ConnectorSourceWorker 🔒
  ConnectorSourceWorker keeps fetching the latest split metadata from the external source service (Self::tick), and maintains it in current_splits.
• ConnectorSourceWorkerHandle
  Handle for a running ConnectorSourceWorker.
• SharedSplitMap 🔒
• SourceManager
  SourceManager keeps fetching the latest split metadata from the external source services (ConnectorSourceWorker::tick), and sends a split assignment command if split changes detected (Self::tick).
• SourceManagerCore
• SourceManagerRunningInfo
• SplitDiffOptions 🔒

Constants

• DEFAULT_SOURCE_TICK_TIMEOUT 🔒
• DEFAULT_SOURCE_WORKER_TICK_INTERVAL 🔒
• MAX_FAIL_CNT 🔒

Functions

• align_backfill_splits 🔒
• build_actor_connector_splits
• build_actor_split_impls
• create_source_worker_handle
  Used to create a new ConnectorSourceWorkerHandle for a new source.
• extract_prop_from_existing_source 🔒
• extract_prop_from_new_source 🔒
• reassign_splits 🔒
  Reassigns splits if there are new splits or dropped splits, i.e., actor_splits and discovered_splits differ, or actors are rescheduled.

Type Aliases

• SharedSplitMapRef 🔒
• SourceManagerRef
• SplitAssignment
• ThrottleConfig