Team removal results in significant space unbalance
Open
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
The data distribution team removal procedure gets run when changing the machines present in a cluster (for example, by exclusion/inclusion, adding new machines, etc.). When it happens, it often seems to result in a significant imbalance in the number of bytes stored on different processes.
This is a problem because some of the processes end up storing significantly more than they had been previously (one example I saw was 25% more for the worst process), which may not be easily accommodated in fuller clusters.
This is eventually healed after the team removal is complete and rebalancing movement can correct the problem.
Contributor guide
Assessment
This issue has not been assessed yet.