Improve data distribution balancing with unequal number of processes per zoneId
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
EBay ran into a situation where a cluster ran out of disk space because FDB was not able to properly balance data across the storage servers. This was because although they had 8 different zoneIds, some of those zones only had a few processes and others had 10+ processes.
It should be possible for FDB to keep disk space balanced in these scenarios, however it appears the existing team logic does not work for this scenario.
Contributor guide
Research direction
Start by reproducing the reported imbalance with eight zoneIds and unequal numbers of processes, then trace the existing team logic responsible for data distribution. Confirm the failure mode by checking storage-server disk usage across zones. Done means data remains balanced when zones contain different numbers of processes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100