facebook / facebook/rocksdb

drop deletion markers in compaction when they don't cover any data

Open
#3,842 7 comments 0 reactions 0 assignees View on GitHub
enhancement up-for-grabs
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

When compacting to bottom level (the level in the LSM containing the oldest data) we currently output deletion markers ("tombstones") if they're newer than any snapshot. That's because the key covered by a tombstone may still be visible to a snapshot, so we can't drop either it or its tombstone yet.

One time, this caused an issue for MyRocks where there were a huge number of tombstones output by compaction even though the keys they covered had already been dropped (they were either non-existent initially or no longer visible to any snapshots). We should make our check more precise and only write tombstones out if they also actually cover some real key, not just check if it's theoretically possible for them to cover a key. This will require doing some lookahead on the compaction iterator to check what a tombstone covers before deciding whether to output it or not.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.