facebook / facebook/rocksdb

Mempurge instead of flush is initiated even if only one memtable is picked by flush job

Open
#9,151 4 comments 0 reactions 1 assignee Claimed by @briantkim93 View on GitHub
bug
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

This can be a potential bug once we merge #9142, which is a fix for a legit bug causing DB::Open failure. Currently before the fix, this bug is hidden.

In https://github.com/facebook/rocksdb/blob/6.26.fb/db/flush_job.cc#L233, a flush job will initiate a mempurge instead of flush even if `mems_.size()` is 1. Consequently, this flush job does not reduce the number of immutable memtables, leading to higher chance of write stall.

### Expected behavior
When the number of immutable memtables reaches threshold, a flush is scheduled and executed, resulting in reduced number of immutable memtables. The db will eventually get out of write-stall, even when there are a lot of writes.

### Actual behavior
Currently, when the number of immutable reaches threshold, a mempurge may be scheduled even if the number of memtables picked is 1. The new memtable will be added back, and does not mitigate write-stall condition. No further flush may be scheduled because normally a flush is scheduled after insertion, but insertion is currently stalled.

### Steps to reproduce the behavior
Use #9150 , restart the job "build-linux-non-shm-1" with ssh access. Manually run the following
```
./db_flush_test --gtest_filter=DBFlushTest.MemPurgeWALSupport
```
It will hang.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.