cockroachdb / cockroachdb/cockroach
kvserver: diversityRebalanceFromScore can be expensive
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Describe the problem**
Periodically, we iterate over all replicas in a store and call shouldQueue. If no other actions are needed, we fall back to checking for potential rebalances via RebalanceVoter and RebalanceNonVoter. These, in turn, call rankedCandidateListForRebalancing, which iterates over all stores in the cluster to generate candidates for every existing store. For each, it computes a diversity score as if it is removing the replica from the current store and adding it to the candidate. When computing the diversity score, it performs a pairwise locality comparison across existing stores, which is an O(existing store count ^2).
There seems to be a potential optimization here where we can cache pairwise diversity score computation somehow - assuming localities of stores don't change often (https://github.com/cockroachdb/cockroach/blob/1942004afe7af1c4b786c2147544e71e947b61a6/pkg/kv/kvserver/allocator/allocatorimpl/allocator_scorer.go#L2332).
MMA already does optimization like this: take a look at diversityScoringMemo and existingReplicaLocalities.
---
Since MMA already has this optimization, I'm not sure if we want to work on this.
Jira issue: CRDB-51251
Contributor guide
Assessment
This issue has not been assessed yet.