defrag stops the process completely for noticeable duration
- Dominant language
- Go
- Stars
- 52.3k
- Forks
- 10.5k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 43
Description
# What I observed
The defrag command blocks all the requests handled by the process for a while when the process has much data and incoming requests.
New requests and ongoing streaming requests such as `Watch()` can be canceled by their deadlines and the cluster can be unstable.
# Side effects
This "stop the world" behavior makes the operation of etcd clusters harder. We have the 8GB limitation of the DB size and need to control the DB size not to reach the border. However, clusters won't release DB space even after compaction until you run the defrag command. That means you cannot track this important metric without virtually stopping every etcd processe periodically.
(Btw, can we add a new metric for "actual DB usage"?)
CC/ @xiang90
Contributor guide
Research direction
Start by reproducing the defrag command while the process handles incoming and streaming requests such as Watch(). Investigate how defragmentation affects request handling and whether actual database usage can be measured. Done should mean defragmentation no longer causes unacceptable request disruption, with any new usage metric defined and validated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100