Module hash

Modules

• consistent_hash 🔒
• dispatcher 🔒
• key 🔒
  This module contains implementation for hash key serialization for hash-agg, hash-join, and perhaps other hash-based operators.
• key_v2 🔒
• marker
• table_distribution

Structs

• HashCode
  A wrapper for u64 hash result. Generic over the hasher.
• HashKeyImpl
  The implementation of the hash key.
• HeapNullBitmap
  Null bitmap on heap. We use this for the edge case where group key sizes are larger than 64. This is because group key null bits cannot fit into a u64 on the stack if they exceed 64 bits. NOTE(kwannoel): This is not really optimized as it is an edge case.
• HeapStorage
  Key storage that uses a in-heap buffer and key, backed by a boxed slice.
• PrecomputedBuildHasher
• PrecomputedHasher
  A special hasher designed for HashKey, which stores a hash key from HashKey::hash() and outputs it on finish().
• StackBuffer
  The buffer for building a hash key on a fixed-size byte array on the stack.
• StackNullBitmap
  Null Bitmap on stack. This is specialized for the common case where group keys (<= 64).
• StackStorage
  Key storage that uses a on-stack buffer and key, backed by a byte array with length N.
• VirtualNode
  VirtualNode (a.k.a. Vnode) is a minimal partition that a set of keys belong to. It is used for consistent hashing.
• VnodeMapping
  Generic mapping from virtual nodes to items.
• WorkerSlotId

Enums

• VnodeCount
  The different cases of maybe_vnode_count field in the protobuf message.

Statics

• MAX_GROUP_KEYS_ON_STACK
  This is determined by the stack based data structure we use, StackNullBitmap, which can store 64 bits at most.

Traits

• Buffer
  Associated type for KeyStorage used to build the hash key.
• HashKey
  Trait for different kinds of hash keys.
• HashKeyDe
  The deserialization counterpart of HashKeySer.
• HashKeyDispatcher
  A trait to help to dynamically dispatch HashKey template.
• HashKeySer
  Extension of scalars to be serialized into hash keys.
• IsSingleton
  A trait for checking whether a table/fragment is a singleton.
• KeyStorage
  The storage where the hash key resides in memory.
• NullBitmap
  We use a trait for NullBitmap so we can parameterize structs on it. This is because NullBitmap is used often, and we want it to occupy the minimal stack space.
• VnodeBitmapExt
  An extension trait for Bitmap to support virtual node operations.
• VnodeCountCompat
  A trait for accessing the vnode count field with backward compatibility.
• VnodeMappingItem
  Trait for items that can be used as keys in VnodeMapping.

Functions

• calc_hash_key_kind
  Calculate what kind of hash key should be used given the key data types.

Type Aliases

• ActorId
• ActorMapping
  A mapping from VirtualNode to ActorId.
• AllVirtualNodeIter
  An iterator over all virtual nodes.
• Crc32HashCode
  Hash code from the Crc32 hasher. Used for hash-shuffle exchange.
• ExpandedActorMapping
  An expanded mapping from VirtualNode to ActorId.
• ExpandedMapping
  Exapnded mapping from virtual nodes to items, essentially a vector of items and can be indexed by virtual nodes.
• ExpandedWorkerSlotMapping
  An expanded mapping from VirtualNode to WorkerSlotId.
• FixedSizeKey
• Key8
• Key16
• Key32
• Key64
• Key128
• Key256
• KeySerialized
• SerializedKey
• WorkerSlotMapping
  A mapping from VirtualNode to WorkerSlotId.
• XxHash64HashCode
  Hash code from the XxHash64 hasher. Used for in-memory hash map cache.