grafana / grafana/pyroscope

Reject compaction jobs for deleted blocks

Open
#5,436 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
11.7k
Forks
802
Avg merge
1d 19h
Merged PRs (30d)
80

Description

In a rare case, either due to configuring a very short retention period or deploying an undersized compaction worker pool, v2 compaction workers could compact blocks that were removed due to retention. This can resurrect data that was deleted.

Add checks before `ReplaceBlocks` that all blocks in `compacted.SourceBlocks` still exist. On mismatch, skip `ReplaceBlocks` and tombstone the output block that was already written. Mark the job as completed.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the v2 compaction worker and the ReplaceBlocks call, then trace how compacted.SourceBlocks and the already-written output block are handled. Done means missing source blocks prevent ReplaceBlocks, the output block is tombstoned, and the job is marked completed.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability-sre
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.