Struct CapacitySplitTableBuilder

pub struct CapacitySplitTableBuilder<F>
where
    F: TableBuilderFactory,
{

    builder_factory: F,
    sst_outputs: Vec<LocalSstableInfo>,
    current_builder: Option<SstableBuilder<F::Writer, F::Filter>>,
    pub compactor_metrics: Arc<CompactorMetrics>,
    task_progress: Option<Arc<TaskProgress>>,
    last_table_id: u32,
    vnode_count: usize,
    table_vnode_partition: BTreeMap<u32, u32>,
    split_weight_by_vnode: u32,
    largest_vnode_in_current_partition: usize,
    concurrent_upload_join_handle: FuturesUnordered<UploadJoinHandle>,
    concurrent_uploading_sst_count: Option<usize>,
    compaction_catalog_agent_ref: CompactionCatalogAgentRef,
}

A wrapper for SstableBuilder which automatically split key-value pairs into multiple tables, based on their target capacity set in options.

When building is finished, one may call finish to get the results of zero, one or more tables.

Fields

builder_factory: F

When creating a new SstableBuilder, caller use this factory to generate it.

sst_outputs: Vec<LocalSstableInfo>

current_builder: Option<SstableBuilder<F::Writer, F::Filter>>

compactor_metrics: Arc<CompactorMetrics>

Statistics.

task_progress: Option<Arc<TaskProgress>>

Update the number of sealed Sstables.

last_table_id: u32

vnode_count: usize

table_vnode_partition: BTreeMap<u32, u32>

split_weight_by_vnode: u32

largest_vnode_in_current_partition: usize

When vnode of the coming key is greater than largest_vnode_in_current_partition, we will switch SST.

concurrent_upload_join_handle: FuturesUnordered<UploadJoinHandle>

concurrent_uploading_sst_count: Option<usize>

compaction_catalog_agent_ref: CompactionCatalogAgentRef

Implementations

impl<F> CapacitySplitTableBuilder<F>

where F: TableBuilderFactory,

pub fn new( builder_factory: F, compactor_metrics: Arc<CompactorMetrics>, task_progress: Option<Arc<TaskProgress>>, table_vnode_partition: BTreeMap<u32, u32>, concurrent_uploading_sst_count: Option<usize>, compaction_catalog_agent_ref: CompactionCatalogAgentRef, ) -> Self

Creates a new CapacitySplitTableBuilder using given configuration generator.

pub fn for_test( builder_factory: F, compaction_catalog_agent_ref: CompactionCatalogAgentRef, ) -> Self

pub fn len(&self) -> usize

Returns the number of SstableBuilders.

pub fn is_empty(&self) -> bool

Returns true if no builder is created.

pub async fn add_full_key_for_test( &mut self, full_key: FullKey<&[u8]>, value: HummockValue<&[u8]>, is_new_user_key: bool, ) -> HummockResult<()>

pub async fn add_raw_block( &mut self, buf: Bytes, filter_data: Vec<u8>, smallest_key: FullKey<Vec<u8>>, largest_key: Vec<u8>, block_meta: BlockMeta, ) -> HummockResult<bool>

pub async fn add_full_key( &mut self, full_key: FullKey<&[u8]>, value: HummockValue<&[u8]>, is_new_user_key: bool, ) -> HummockResult<()>

Adds a key-value pair to the underlying builders.

If allow_split and the current builder reaches its capacity, this function will create a new one with the configuration generated by the closure provided earlier.

Note that in some cases like compaction of the same user key, automatic splitting is not allowed, where allow_split should be false.

pub fn check_switch_builder(&mut self, user_key: &UserKey<&[u8]>) -> bool

pub fn need_flush(&self) -> bool

pub async fn seal_current(&mut self) -> HummockResult<()>

Marks the current builder as sealed. Next call of add will always create a new table.

If there’s no builder created, or current one is already sealed before, then this function will be no-op.

pub async fn finish(self) -> HummockResult<Vec<LocalSstableInfo>>

Finalizes all the tables to be ids, blocks and metadata.