cockroachdb / cockroachdb/pebble
compact: consider value separation during heuristic for sstable splitting
- Dominant language
- Go
- Stars
- 6k
- Forks
- 584
- Avg merge
- 16h 35m
- Merged PRs (30d)
- 5
Description
Currently, during compaction, we split sstables based on `targetFileSize`. When value separation is enabled, the size of an sstable's blob references is included in this calculation:
https://github.com/annrpom/pebble/blob/edebc31de6b06c32fa37279927d6c7f2c476cfed/internal/compact/run.go#L283-L287
In a one-off test with an LSM containing ~4 TiB of separated values, the average sstable size seemed unexpectedly small (~60 KiB):
We should update our sstable-splitting logic to better account for aggressive value separation (at the very least) to see if we can bias less towards smaller sstables
Jira issue: PEBBLE-1142
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.