cockroachdb / cockroachdb/pebble

db: prefer compacting into L6 tables with many tombstones

Open
#4,575 0 comments 0 reactions 0 assignees View on GitHub
A-storage T-storage
Dominant language
Go
Stars
6k
Forks
584
Avg merge
16h 35m
Merged PRs (30d)
5

Description

When picking a file to compact into L6, we consider the L6 file's `RangeDeletionBytesEstimate`, prioritizing compactions that will compact L6 files with a high estimate:
https://github.com/cockroachdb/pebble/blob/431a23b680c1305b289530c9f7a33dd5bf28b5d4/compaction_picker.go#L1096-L1107

We do this because when we populate `RangeDeletionBytesEstimate`, we'll estimate the size within the file itself to capture the fact that the range deletions delete data within the file itself (and presumably still exist due to LSM snapshots that were open when the file was created):

https://github.com/cockroachdb/pebble/blob/431a23b680c1305b289530c9f7a33dd5bf28b5d4/table_stats.go#L422-L424

The compaction picking logic does not take into account `PointDeletionBytesEstimate`. The `PointDeletionBytesEstimate` doesn't have any special logic accounting for L6 sstables. My reading of the code is that it already includes the size of the tombstones themselves in the estimate, so it's already optimistic in its calculation.

We should update the compaction picking logic to also consider the PointDeletionBytesEstimate when computing the min-overlapping ratio of a compaction.

Jira issue: PEBBLE-411

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.