Module hash

Modules§

consistent_hash 🔒
dispatcher 🔒
key 🔒: This module contains implementation for hash key serialization for hash-agg, hash-join, and perhaps other hash-based operators.
key_v2 🔒
marker
table_distribution

HashCode: A wrapper for u64 hash result. Generic over the hasher.
HashKeyImpl: The implementation of the hash key.
HeapNullBitmap: Null bitmap on heap. We use this for the edge case where group key sizes are larger than 64. This is because group key null bits cannot fit into a u64 on the stack if they exceed 64 bits. NOTE(kwannoel): This is not really optimized as it is an edge case.
HeapStorage: Key storage that uses a in-heap buffer and key, backed by a boxed slice.
PrecomputedBuildHasher
PrecomputedHasher: A special hasher designed for HashKey, which stores a hash key from HashKey::hash() and outputs it on finish().
StackBuffer: The buffer for building a hash key on a fixed-size byte array on the stack.
StackNullBitmap: Null Bitmap on stack. This is specialized for the common case where group keys (<= 64).
StackStorage: Key storage that uses a on-stack buffer and key, backed by a byte array with length N.
VirtualNode: VirtualNode (a.k.a. Vnode) is a minimal partition that a set of keys belong to. It is used for consistent hashing.
VnodeMapping: Generic mapping from virtual nodes to items.
WorkerSlotId

VnodeCount: The different cases of maybe_vnode_count field in the protobuf message.

MAX_GROUP_KEYS_ON_STACK: This is determined by the stack based data structure we use, StackNullBitmap, which can store 64 bits at most.

Buffer: Associated type for KeyStorage used to build the hash key.
HashKey: Trait for different kinds of hash keys.
HashKeyDe: The deserialization counterpart of HashKeySer.
HashKeyDispatcher: A trait to help to dynamically dispatch HashKey template.
HashKeySer: Extension of scalars to be serialized into hash keys.
IsSingleton: A trait for checking whether a table/fragment is a singleton.
KeyStorage: The storage where the hash key resides in memory.
NullBitmap: We use a trait for NullBitmap so we can parameterize structs on it. This is because NullBitmap is used often, and we want it to occupy the minimal stack space.
VnodeBitmapExt: An extension trait for Bitmap to support virtual node operations.
VnodeCountCompat: A trait for accessing the vnode count field with backward compatibility.
VnodeMappingItem: Trait for items that can be used as keys in VnodeMapping.

calc_hash_key_kind: Calculate what kind of hash key should be used given the key data types.

ActorId
ActorMapping: A mapping from VirtualNode to ActorId.
AllVirtualNodeIter: An iterator over all virtual nodes.
Crc32HashCode: Hash code from the Crc32 hasher. Used for hash-shuffle exchange.
ExpandedActorMapping: An expanded mapping from VirtualNode to ActorId.
ExpandedMapping: Exapnded mapping from virtual nodes to items, essentially a vector of items and can be indexed by virtual nodes.
ExpandedWorkerSlotMapping: An expanded mapping from VirtualNode to WorkerSlotId.
FixedSizeKey
Key8
Key16
Key32
Key64
Key128
Key256
KeySerialized
SerializedKey
WorkerSlotMapping: A mapping from VirtualNode to WorkerSlotId.
XxHash64HashCode: Hash code from the XxHash64 hasher. Used for in-memory hash map cache.