risingwave_meta::stream

Function rebalance_actor_vnode

source
pub fn rebalance_actor_vnode(
    actors: &[CustomActorInfo],
    actors_to_remove: &BTreeSet<ActorId>,
    actors_to_create: &BTreeSet<ActorId>,
) -> HashMap<ActorId, Bitmap>
Expand description

This function provides an simple balancing method The specific process is as follows

  1. Calculate the number of target actors, and calculate the average value and the remainder, and use the average value as expected.

  2. Filter out the actor to be removed and the actor to be retained, and sort them from largest to smallest (according to the number of virtual nodes held).

  3. Calculate their balance, 1) For the actors to be removed, the number of virtual nodes per actor is the balance. 2) For retained actors, the number of virtual nodes - expected is the balance. 3) For newly created actors, -expected is the balance (always negative).

  4. Allocate the remainder, high priority to newly created nodes.

  5. After that, merge removed, retained and created into a queue, with the head of the queue being the source, and move the virtual nodes to the destination at the end of the queue.

This can handle scale in, scale out, migration, and simultaneous scaling with as much affinity as possible.

Note that this function can only rebalance actors whose vnode_bitmap is not None, in other words, for Fragment of FragmentDistributionType::Single, using this function will cause assert to fail and should be skipped from the upper level.

The return value is the bitmap distribution after scaling, which covers all virtual node indexes