Data movement can't begin on clusters missing data
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
When a cluster loses all replicas of a shard and the data distributor later restarts, it gets stuck trying to track the initial shards (Note: I'm not certain if this is universally true or if it requires other properties to hold). As a result, no data movement can happen with the data that still exists in the cluster.
It would be better if data movement could continue on the shards that remain, which could help us to prevent increasing the blast radius of this failure case in some circumstances.
Contributor guide
Research direction
No files, tests, or entry points are named. Begin by tracing the data distributor's restart and initial-shard tracking path, then reproduce the missing-replica scenario; done means data movement continues for surviving shards after restart.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100