Struct IcebergTaskQueue 

pub struct IcebergTaskQueue {
    inner: IcebergTaskQueueInner,
    max_parallelism: u32,
    pending_parallelism_budget: u32,
    schedule_notify: Option<Arc<Notify>>,
}

FIFO compaction task queue with lightweight replacement and parallelism budgeting. Features:

• Push with replacement (same unique_ident waiting -> metadata/runner updated in place).
• Pop moves the head (if it fits remaining parallelism) to the running set.
• Reject duplicate if the same unique_ident is already running.
• Capacity control: reject when adding/replacing would exceed pending_parallelism_budget (sum of waiting tasks’ required_parallelism). No eviction beyond hard rejection.
• Optional notification when new tasks become schedulable.

Invariants (enforced by logic; violation triggers panic):

• waiting ∩ running = ∅.
• Each waiting unique_ident appears exactly once in deque.
• waiting_parallelism_sum = Σ required_parallelism(waiting).
• running_parallelism_sum = Σ required_parallelism(running).

Fields

inner: IcebergTaskQueueInner

max_parallelism: u32

Maximum parallelism that a single task may require (cluster effective max / scheduling window upper bound).

pending_parallelism_budget: u32

Budget for sum(required_parallelism) of waiting tasks (buffer), e.g. 4 * max_parallelism.

schedule_notify: Option<Arc<Notify>>

Notification for when tasks become schedulable

Implementations

impl IcebergTaskQueue

pub fn new(max_parallelism: u32, pending_parallelism_budget: u32) -> Self

pub fn new_with_notify( max_parallelism: u32, pending_parallelism_budget: u32, ) -> (Self, Arc<Notify>)

pub async fn wait_schedulable(&self) -> bool

fn has_schedulable_tasks(&self) -> bool

fn notify_schedulable(&self)

pub fn running_parallelism_sum(&self) -> u32

pub fn waiting_parallelism_sum(&self) -> u32

fn available_parallelism(&self) -> u32

pub fn push( &mut self, meta: IcebergTaskMeta, runner: Option<IcebergCompactorRunner>, ) -> PushResult

Enqueue semantics:

• Waiting duplicate: replace in place (position preserved) -> Replaced { old_task_id }.
  • If new runner is Some, replace runner; else keep old runner.
• Running duplicate: reject -> RejectedRunningDuplicate.
• Budget exceeded (sum of waiting parallelism would surpass pending_parallelism_budget): RejectedCapacity.
• required_parallelism == 0: RejectedInvalidParallelism.
• required_parallelism > max_parallelism: RejectedTooLarge.
• Otherwise append -> Added.

pub fn pop(&mut self) -> Option<PoppedIcebergTask>

pub fn finish_running(&mut self, task_id: u64) -> bool