cockroachdb / cockroachdb/pebble

db: consider biasing towards rewriting blob files with few references

Open
#4,763 0 comments 0 reactions 0 assignees View on GitHub
A-storage T-storage
Dominant language
Go
Stars
6k
Forks
584
Avg merge
16h 35m
Merged PRs (30d)
5

Description

Currently the MaxBlobReferenceDepth heuristic drives the decision of whether or not a compaction should write values to new blob files. This heuristic doesn't take into account the possibility that writing new blob files may allow disk space reclamation. It only prioritizes restoring locality.

In the extreme case, imagine a compaction with 1 input sstable referencing 10KiB of a single 128 MiB blob file. If all the other values in the blob file are unreferenced, the blob file only remains because of the extant references that the compaction is rewriting. Even if the MaxBlobReferenceDepth heuristic wouldn't otherwise choose to write new blob files, reclaiming the ~128MiB is worth the small amount of additional write I/O from writing the 10KiB values to a new blob file.

Jira issue: PEBBLE-453

Epic PEBBLE-1226

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.