facebook / facebook/rocksdb

Can RocksDB figure out whether or not to compress flushes WRT trivial move

Open
#9,292 1 comment 1 reaction 0 assignees View on GitHub
enhancement up-for-grabs
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

> Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://groups.google.com/forum/#!forum/rocksdb or https://www.facebook.com/groups/rocksdb.dev

### Expected behavior

I want RocksDB to know whether or not to compression flushes based on whether trivial moves will be used.

### Actual behavior

A common pattern for benchmarks and real workloads is load then do something interesting. The load is frequently is key order and benefits from trivial moves to reduce compaction write-amp. However, this is difficult to get right WRT to compression. The problem is that during the load phase the memtable flushes should be compressed as they might be trivial moved down the LSM tree to a level that should be compressed. But during the "do something interesting" phase the flushes usually should not be compressed as compressing small levels (sorted runs) uses a lot of CPU without saving much space.

Today I have to statically configure RocksDB to compress flushes during the load, then change the config and restart RocksDB or MyRocks to not compress flushes during the "do something interesting" phase. I prefer that RocksDB figure this out without guidance other than the values for the level at which compression starts (for leveled) or the value of compression_size_percent (for universal). I prefer to not have to edit the config and restart RocksDB (or MyRocks) after the load phase.

### Steps to reproduce the behavior

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.