facebook / facebook/rocksdb

[Proposal]Optimize OpenAndTrimHistory performance with skip unnessary compacting files by checking each file's timestamp meta

Open
#9,863 0 comments 0 reactions 0 assignees View on GitHub
question up-for-grabs
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

A new API named DB::OpenAndTrimHistory is added to imported by (https://github.com/facebook/rocksdb/issues/9223). This API will open DB and trim data to the timestamp specified by trim_ts (The data with newer timestamp than specified trim bound will be removed). Currently this API will iterate every sst file to trim data which has newer timestamp than specified trim_ts. Actually files with max_ts smaller than trim_ts should be safe and no need to be iterated.

Since we can track sst file's max and min timestamp according (https://github.com/facebook/rocksdb/issues/8959),
I think the OpenAndTrimHistory API can be optimized by skipping unnessary sst files with call API CompactFiles instead of CompactRangeInternal.

@riversand963

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.