cockroachdb / cockroachdb/pebble

internal/compact: DeletableValueMerger needDelete path corrupts iterator state

Open
#6,168 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
6k
Forks
584
Avg merge
16h 35m
Merged PRs (30d)
5

Description

## Summary

When a merger implementing `base.DeletableValueMerger` returns `delete=true` from `DeletableFinish` during a compaction, the compaction iterator's `needDelete` handling elides the merge result via `continue` without performing the `skip/position` bookkeeping that the normal returned-key path performs. Depending on arrangement this causes either:

- Silent data corruption: the just-consumed base SET (and any older same-key entries the pending skip was supposed to elide) are re-processed as fresh keys, resurrecting a value for a key the merger declared non-existent; or
- A panic: compaction iterator has skip=true, but iterator is at iterPosNext on a subsequent Iter.Next() call.

Reproduced on `v2.1.4`, `v2.1.5`, `v2.1.6`; the relevant code is unchanged on master.

Jira issue: PEBBLE-1469

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in internal/compact at the compaction iterator's needDelete handling and trace the DeletableFinish path alongside the normal returned-key path. Reproduce the report on v2.1.4–v2.1.6 or master, then verify that skip/position bookkeeping remains consistent, with no resurrected values or iterPosNext panic.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.