[Proposal]Optimize OpenAndTrimHistory performance with skip unnessary compacting files by checking each file's timestamp meta
- Dominant language
- C++
- Stars
- 32.1k
- Forks
- 6.9k
- Avg merge
- 32m
- Merged PRs (30d)
- 1
Description
A new API named DB::OpenAndTrimHistory is added to imported by (https://github.com/facebook/rocksdb/issues/9223). This API will open DB and trim data to the timestamp specified by trim_ts (The data with newer timestamp than specified trim bound will be removed). Currently this API will iterate every sst file to trim data which has newer timestamp than specified trim_ts. Actually files with max_ts smaller than trim_ts should be safe and no need to be iterated.
Since we can track sst file's max and min timestamp according (https://github.com/facebook/rocksdb/issues/8959),
I think the OpenAndTrimHistory API can be optimized by skipping unnessary sst files with call API CompactFiles instead of CompactRangeInternal.
@riversand963
Contributor guide
Assessment
This issue has not been assessed yet.