Reject compaction jobs for deleted blocks
- Dominant language
- Go
- Stars
- 11.7k
- Forks
- 802
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 80
Description
In a rare case, either due to configuring a very short retention period or deploying an undersized compaction worker pool, v2 compaction workers could compact blocks that were removed due to retention. This can resurrect data that was deleted.
Add checks before `ReplaceBlocks` that all blocks in `compacted.SourceBlocks` still exist. On mismatch, skip `ReplaceBlocks` and tombstone the output block that was already written. Mark the job as completed.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the v2 compaction worker and the ReplaceBlocks call, then trace how compacted.SourceBlocks and the already-written output block are handled. Done means missing source blocks prevent ReplaceBlocks, the output block is tombstoned, and the job is marked completed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 64/100