cockroachdb / cockroachdb/pebble
db: prefer compacting into L6 tables with many tombstones
- Dominant language
- Go
- Stars
- 6k
- Forks
- 584
- Avg merge
- 16h 35m
- Merged PRs (30d)
- 5
Description
When picking a file to compact into L6, we consider the L6 file's `RangeDeletionBytesEstimate`, prioritizing compactions that will compact L6 files with a high estimate:
https://github.com/cockroachdb/pebble/blob/431a23b680c1305b289530c9f7a33dd5bf28b5d4/compaction_picker.go#L1096-L1107
We do this because when we populate `RangeDeletionBytesEstimate`, we'll estimate the size within the file itself to capture the fact that the range deletions delete data within the file itself (and presumably still exist due to LSM snapshots that were open when the file was created):
https://github.com/cockroachdb/pebble/blob/431a23b680c1305b289530c9f7a33dd5bf28b5d4/table_stats.go#L422-L424
The compaction picking logic does not take into account `PointDeletionBytesEstimate`. The `PointDeletionBytesEstimate` doesn't have any special logic accounting for L6 sstables. My reading of the code is that it already includes the size of the tombstones themselves in the estimate, so it's already optimistic in its calculation.
We should update the compaction picking logic to also consider the PointDeletionBytesEstimate when computing the min-overlapping ratio of a compaction.
Jira issue: PEBBLE-411
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.