Struct SstableStreamIterator

pub struct SstableStreamIterator {

    sstable_store: SstableStoreRef,
    block_metas: Vec<BlockMeta>,
    block_stream: Option<BlockDataStream>,
    block_iter: Option<BlockIterator>,
    block_idx: usize,
    stats_ptr: Arc<AtomicU64>,
    sstable_info: SstableInfo,
    sstable_table_ids: HashSet<StateTableId>,
    task_progress: Arc<TaskProgress>,
    io_retry_times: usize,
    max_io_retry_times: usize,
    key_range_left: FullKey<Vec<u8>>,
    key_range_right: FullKey<Vec<u8>>,
    key_range_right_exclusive: bool,
}

Iterates over the KV-pairs of an SST while downloading it. SstableStreamIterator encapsulates operations on sstables, constructing block streams and accessing the corresponding data via block_metas. Note that a block_meta does not necessarily correspond to the entire sstable, but rather to a subset, which is documented via the block_idx.

Fields

sstable_store: SstableStoreRef

block_metas: Vec<BlockMeta>

The block metas subset of the SST.

block_stream: Option<BlockDataStream>

The downloading stream.

block_iter: Option<BlockIterator>

Iterates over the KV-pairs of the current block.

block_idx: usize

Index of the current block.

stats_ptr: Arc<AtomicU64>

Counts the time used for IO.

sstable_info: SstableInfo

For key sanity check of divided SST and debugging

sstable_table_ids: HashSet<StateTableId>

To Filter out the blocks

task_progress: Arc<TaskProgress>

io_retry_times: usize

max_io_retry_times: usize

key_range_left: FullKey<Vec<u8>>

key_range_right: FullKey<Vec<u8>>

key_range_right_exclusive: bool

Implementations

impl SstableStreamIterator

pub fn new( block_metas: Vec<BlockMeta>, sstable_info: SstableInfo, stats: &StoreLocalStatistic, task_progress: Arc<TaskProgress>, sstable_store: SstableStoreRef, max_io_retry_times: usize, ) -> Self

Initialises a new SstableStreamIterator which iterates over the given BlockDataStream. The iterator reads at most max_block_count from the stream.

async fn create_stream(&mut self) -> HummockResult<()>

async fn prune_from_valid_block_iter(&mut self) -> HummockResult<()>

pub async fn seek( &mut self, seek_key: Option<FullKey<&[u8]>>, ) -> HummockResult<()>

Initialises the iterator by moving it to the first KV-pair in the stream’s first block where key >= seek_key. If that block does not contain such a KV-pair, the iterator continues to the first KV-pair of the next block. If seek_key is not given, the iterator will move to the very first KV-pair of the stream’s first block.

async fn next_block(&mut self) -> HummockResult<()>

Loads a new block, creates a new iterator for it, and stores that iterator in self.block_iter. The created iterator points to the block’s first KV-pair. If the end of the stream is reached or self.remaining_blocks is zero, then the function sets self.block_iter to None.

pub async fn next(&mut self) -> HummockResult<()>

Moves to the next KV-pair in the table. Assumes that the current position is valid. Even if the next position is invalid, the function returns Ok(()).

Do not use next() to initialise the iterator (i.e. do not use it to find the first KV-pair). Instead, use seek(). Afterwards, use next() to reach the second KV-pair and onwards.

pub fn key(&self) -> FullKey<&[u8]>

pub fn value(&self) -> HummockValue<&[u8]>

pub fn is_valid(&self) -> bool

fn sst_debug_info(&self) -> String

fn need_recreate_io_stream(&self) -> bool

fn exceed_key_range_left(&self, key: FullKey<&[u8]>) -> bool

fn exceed_key_range_right(&self, key: FullKey<&[u8]>) -> bool

Trait Implementations

impl Drop for SstableStreamIterator

fn drop(&mut self)

Executes the destructor for this type.