Skip to main content

Module worker

Module worker 

Source

Structsยง

ConnectorSourceWorker
ConnectorSourceWorker keeps fetching the latest split metadata from the external source service (Self::tick), and maintains it in current_splits.
ConnectorSourceWorkerHandle
Handle for a running ConnectorSourceWorker.
SharedSplitMap

Enumsยง

SourceWorkerCommand

Constantsยง

DEBUG_SPLITS_KEY ๐Ÿ”’
DEFAULT_SOURCE_WORKER_TICK_INTERVAL ๐Ÿ”’
MAX_FAIL_CNT ๐Ÿ”’
TICK_INFLIGHT_WARN_INTERVAL ๐Ÿ”’
Interval at which a still-running (in-flight) source worker tick emits a progressive warning.

Functionsยง

create_source_worker
Used to create a new ConnectorSourceWorkerHandle for a new source.
create_source_worker_async
Used on startup (SourceManager::new). Failed sources will not block meta startup.
extract_prop_from_existing_source ๐Ÿ”’
extract_prop_from_new_source ๐Ÿ”’
run_tick_with_progressive_warn ๐Ÿ”’
Drive a source worker tick future to completion, recording its duration in tick_duration and emitting a warn! every TICK_INFLIGHT_WARN_INTERVAL while it is still pending. The future is never cancelled or timed out, so tick() semantics are unchanged.

Type Aliasesยง

SharedSplitMapRef ๐Ÿ”’