When all storage replicas of a shard are unavailable, shall we throttle the write traffic to the shard?
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
When all storage replicas of a shard are unavailable, read to the shard will be unavailable, which is correct behavior.
The current implementation will still accept write transactions until these transactions fill up tLogs' disks, since those unavailable SSes will not pop data.
Since the cluster is already in a bad state when all replicas become unavailable, it may be a better idea to throttle the write traffic to the shard: (1) if the storage replicas are temporarily unavailable (say 10min), we still accept writes; (2) if it's permanently unavailable, we should reject writes to the shard before the cluster gets worse.
Contributor guide
Research direction
The issue names no files, tests, or entry points. Begin by examining how shard storage-replica availability and tLogs disk consumption are handled, then define the temporary-versus-permanent unavailability policy; done requires an agreed design and its implementation, rather than the question currently specifying a concrete change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100