cockroachdb / cockroachdb/pebble

compact: consider value separation during heuristic for sstable splitting

Open
#5,193 5 comments 0 reactions 0 assignees View on GitHub
A-storage T-storage
Dominant language
Go
Stars
6k
Forks
584
Avg merge
16h 35m
Merged PRs (30d)
5

Description

Currently, during compaction, we split sstables based on `targetFileSize`. When value separation is enabled, the size of an sstable's blob references is included in this calculation:

https://github.com/annrpom/pebble/blob/edebc31de6b06c32fa37279927d6c7f2c476cfed/internal/compact/run.go#L283-L287

In a one-off test with an LSM containing ~4 TiB of separated values, the average sstable size seemed unexpectedly small (~60 KiB):

Image

Image

We should update our sstable-splitting logic to better account for aggressive value separation (at the very least) to see if we can bias less towards smaller sstables

Jira issue: PEBBLE-1142

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.