facebook / facebook/rocksdb

why not rocksdb insert a 'VALUE' after the recent 'merge' while 'Get' called?

Open
#11,022 4 comments 1 reaction 0 assignees View on GitHub
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

Hi,

I'm searching a solution to resolve the performance degration of 'get' after about 1000w+ 'merge' operations.

There is a 'max-successive-merge' option indeed, but it will degrade 'merge' performance very obvious, and the larger the 'max_successive_value' setting, the greater the performance degration. Then I find that every 'merge' will do a 'lookup key' inside to get the current merge count of the key, which waste performance obviously.

Why not insert or replace a 'VALUE' when the merge list's length of 'full_merge' reaches 'max_merge'? The 'merge' count of a key doesn't have to limit strictly, cause it designed to balance read and write in essence. In this way, it doesn't have to waste performance in 'lookup key' or repeatedly call 'full_merge' for the same merge list.

Contributor guide

Open the contributing guide

Research direction

Start by tracing RocksDB's merge path and the max-successive-merge setting, focusing on the lookup used to count successive merges and the full_merge operation described in the issue. Compare the current read and write costs after 1000w+ merge operations; done means a reviewed design or implementation that balances merge performance with Get performance without repeated lookups or full merges.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.