db_bench --benchmarks=waitforcompaction hangs with universal compaction
- Dominant language
- C++
- Stars
- 32.1k
- Forks
- 6.9k
- Avg merge
- 32m
- Merged PRs (30d)
- 1
Description
> Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://groups.google.com/forum/#!forum/rocksdb or https://www.facebook.com/groups/rocksdb.dev
### Expected behavior
With --benchmarks=waitforcompaction the test should return once compaction is not in progress or pending.
### Actual behavior
There are intermittent hangs.
### Steps to reproduce the behavior
db_bench --benchmarks=fillseq,overwrite,waitforcompaction
The problem occurs when db_bench [uses kCompactionPending](https://github.com/facebook/rocksdb/blob/dc5de45af8fc1bcc90edea5cd9a5695bd85a8baf/tools/db_bench_tool.cc#L7773) which triggers a [call to HandleCompactionPending](https://github.com/facebook/rocksdb/blob/01bd86ad356a69ab54783aaaa198b6faa5684fb1/db/internal_stats.cc#L453) which [calls NeedsCompaction](https://github.com/facebook/rocksdb/blob/01bd86ad356a69ab54783aaaa198b6faa5684fb1/db/internal_stats.cc#L1033) which [returns true](https://github.com/facebook/rocksdb/blob/c712b68f5bdf2c3ee9b71c6bec599f1451cd806f/db/compaction/compaction_picker_universal.cc#L278) when compaction_score is >= 1 for universal.
This eventually leads to a call to PickCompaction because the read-amp trigger means compaction is needed. But if the sizes of the sorted runs are such that a compaction is not chosen (the next sorted run is too much larger than the current sorted run) then it ends [up here](https://github.com/facebook/rocksdb/blob/c712b68f5bdf2c3ee9b71c6bec599f1451cd806f/db/compaction/compaction_picker_universal.cc#L440) and the problem is the code that does:
```
num_sr_not_compacted >
mutable_cf_options_.level0_file_num_compaction_trigger
```
This only triggers a call to PickCompactionToReduceSortedRuns when the number of sorted runs is greater than L0_file_num_compaction_trigger. However it should be triggered when that number is greater than or equal.
With the current code, we can end up with compaction_score = 1 (because #sorted runs == L0 compaction trigger) but compaction won't get done. So waitforcompaction loops waiting for compaction_score to be less than 1 but that never happens.
Note that if the > is changed to >= then the value of num_files should be incremented by 1
Contributor guide
Assessment
This issue has not been assessed yet.