Skip to main content

row_below_watermark

Function row_below_watermark 

Source
fn row_below_watermark<S: StateStore, SD: ValueRowSerde>(
    row_serde: &BasicSerde,
    table: &StateTableInner<S, SD>,
    row: &CompactedRow,
) -> StreamExecutorResult<bool>
Expand description

Check whether the given row is below the latest watermark of the table. If so, we may want to treat them as non-existent. This extra step is necessary for TTL-ed tables because:

  • the storage may clean expired rows asynchronously
  • an expired row may still be in the cache while we expect a consistent view

The watermark is forwarded downstream once it is fed into the table, so the rows below it are expired before the commit.