Module compactor 

Re-exports

pub use self::compaction_utils::CompactionStatistics;

pub use self::compaction_utils::RemoteBuilderFactory;

pub use self::compaction_utils::TaskConfig;

pub use self::compaction_utils::check_compaction_result;

pub use self::compaction_utils::check_flush_result;

Modules

await_tree_key

compaction_executor 🔒

compaction_filter 🔒

compaction_utils

compactor_runner

context 🔒

fast_compactor_runner

iceberg_compaction 🔒

iterator 🔒

shared_buffer_compact 🔒

task_progress 🔒

Structs

CompactionExecutor

CompactionExecutor is a dedicated runtime for compaction’s CPU intensive jobs.

CompactionLogState 🔒

Represents the compaction task state for logging purposes

Compactor

Implementation of Hummock compaction.

CompactorContext

A CompactorContext describes the context of a compactor.

ConcatSstableIterator

Iterates over the KV-pairs of a given list of SSTs. The key-ranges of these SSTs are assumed to be consecutive and non-overlapping.

DummyCompactionFilter

IcebergCompactionLogState 🔒

Represents the iceberg compaction task state for logging purposes

LogThrottler 🔒

Controls periodic logging with state change detection

MultiCompactionFilter

SstableStreamIterator

Iterates over the KV-pairs of an SST while downloading it. SstableStreamIterator encapsulates operations on sstables, constructing block streams and accessing the corresponding data via block_metas. Note that a block_meta does not necessarily correspond to the entire sstable, but rather to a subset, which is documented via the block_idx.

StateCleanUpCompactionFilter

TaskProgress

The progress of a compaction task.

TtlCompactionFilter

Constants

COMPACTION_HEARTBEAT_LOG_INTERVAL 🔒

Heartbeat logging interval for compaction tasks

Traits

CompactionFilter

Functions

compact

Flush shared buffer to level0. Resulted SSTs are grouped by compaction group.

get_task_progress 🔒

handle_meta_task_pulling 🔒

Handle pulling new tasks from meta service Returns true if the stream should be restarted

merge_imms_in_memory

Merge multiple batches into a larger one

new_compaction_await_tree_reg_ref

schedule_queued_tasks 🔒

Schedule queued tasks if we have capacity

start_compactor

The background compaction thread that receives compaction tasks from hummock compaction manager and runs compaction tasks.

start_iceberg_compactor

The background compaction thread that receives compaction tasks from hummock compaction manager and runs compaction tasks.

start_shared_compactor

The background compaction thread that receives compaction tasks from hummock compaction manager and runs compaction tasks.

Type Aliases

CompactOutput

CompactionAwaitTreeRegRef