facebook / facebook/rocksdb

Enhancement: run L0+L1 to L1 and L1+L2 to L2 compaction concurrently

Open
#11,962 0 comments 0 reactions 0 assignees View on GitHub
feature-request
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

### Expected behavior
`L0+L1 to L1` compaction and `L1+L2 to L2` compactions can running concurrently, just the `Install` needs to be ordered.

For example:
L0: 01 02 03 04 05 06
L1: 11 `12` 13 `14` 15 `16`
L2: 21 `22` 23 `24` 25 `26`

The `files` are compacting in `L1+L2 to L2` compactions, while L0 score are high to compact, the L0+L1 to L1 compaction can start(pick all L0 files and all non-compactiong-L1 files).

If the `L0+L1 to L1` ends before previous `L1+L2 to L2` compactions, the L0+L1 to L1 compaction installation needs wait until previous `L1+L2 to L2` compactions have installed.

### Actual behavior
Currently `L0+L1 to L1` compaction and `L1+L2 to L2` compactions are exclusive, this simplified the compaction scheduling, but the write may be stalled for L0 file number.

### Steps to reproduce the behavior
Write heavy workload.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.