why not rocksdb insert a 'VALUE' after the recent 'merge' while 'Get' called?
- Dominant language
- C++
- Stars
- 32.1k
- Forks
- 6.9k
- Avg merge
- 32m
- Merged PRs (30d)
- 1
Description
Hi,
I'm searching a solution to resolve the performance degration of 'get' after about 1000w+ 'merge' operations.
There is a 'max-successive-merge' option indeed, but it will degrade 'merge' performance very obvious, and the larger the 'max_successive_value' setting, the greater the performance degration. Then I find that every 'merge' will do a 'lookup key' inside to get the current merge count of the key, which waste performance obviously.
Why not insert or replace a 'VALUE' when the merge list's length of 'full_merge' reaches 'max_merge'? The 'merge' count of a key doesn't have to limit strictly, cause it designed to balance read and write in essence. In this way, it doesn't have to waste performance in 'lookup key' or repeatedly call 'full_merge' for the same merge list.
Contributor guide
Research direction
Start by tracing RocksDB's merge path and the max-successive-merge setting, focusing on the lookup used to count successive merges and the full_merge operation described in the issue. Compare the current read and write costs after 1000w+ merge operations; done means a reviewed design or implementation that balances merge performance with Get performance without repeated lookups or full merges.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100