Flaky test DynamicUniversalCompactionReadAmplification
- Dominant language
- C++
- Stars
- 32.1k
- Forks
- 6.9k
- Avg merge
- 32m
- Merged PRs (30d)
- 1
Description
In reference to the test case DynamicUniversalCompactionReadAmplification at:
db/db_universal_compaction_test.cc : 454 to 558
This test case failure was mentioned here (as a result of different changes) and apparently never resolved: https://github.com/facebook/rocksdb/pull/4562
### Expected behavior
I expect `make commit_prereq` to run successfully to completion
The comment in the code states: "Flush whatever is remaining in memtable. This is typically small, about 30KB."
### Actual behavior
"whatever is remaining in memtable" ends up being ~ 90KB (not the 30KB explained in the comment).
The files being written out of memtables have only 8 key/value pairs (as opposed to the expected 10). I'm not sure why the memtables are closer to 80KB in size rather than 100KB because:
options.write_buffer_size = 100 << 10; // 100KB
These test cases are failing.
[ RUN ] UniversalCompactionNumLevels/DBTestUniversalCompaction.DynamicUniversalCompactionReadAmplification/0
db/db_universal_compaction_test.cc:507: Failure
Value of: num + 1
Actual: 3
Expected: NumSortedRuns(1)
Which is: 4
[ FAILED ] UniversalCompactionNumLevels/DBTestUniversalCompaction.DynamicUniversalCompactionReadAmplification/0, where GetParam() = (1, false) (209 ms)
These test cases fail the same way:
[ FAILED ] UniversalCompactionNumLevels/DBTestUniversalCompaction.DynamicUniversalCompactionReadAmplification/0, where GetParam() = (1, false)
[ FAILED ] UniversalCompactionNumLevels/DBTestUniversalCompaction.DynamicUniversalCompactionReadAmplification/1, where GetParam() = (1, true)
[ FAILED ] UniversalCompactionNumLevels/DBTestUniversalCompaction.DynamicUniversalCompactionReadAmplification/2, where GetParam() = (3, false)
[ FAILED ] UniversalCompactionNumLevels/DBTestUniversalCompaction.DynamicUniversalCompactionReadAmplification/3, where GetParam() = (3, true)
[ FAILED ] UniversalCompactionNumLevels/DBTestUniversalCompaction.DynamicUniversalCompactionReadAmplification/4, where GetParam() = (5, false)
[ FAILED ] UniversalCompactionNumLevels/DBTestUniversalCompaction.DynamicUniversalCompactionReadAmplification/5, where GetParam() = (5, true)
### Steps to reproduce the behavior
Run `make commit_prereq` or `make dbg && ./db_universal_compaction_test`
Notes:
Upon further investigation, the "expected" numbers appear to be wrong (and the actual numbers seem to be correct)
Here's the diff that resolves the issue:
` dbfull()->TEST_WaitForFlushMemTable(handles_[1]);`
`- ASSERT_EQ(NumSortedRuns(1), num + 1);`
`+ if (num < 2){ASSERT_EQ(NumSortedRuns(1), num + 1);}`
`+ else{ASSERT_EQ(NumSortedRuns(1), num + 2);}`
` }`
`- ASSERT_EQ(NumSortedRuns(1), options.level0_file_num_compaction_trigger);`
`+ ASSERT_EQ(NumSortedRuns(1), options.level0_file_num_compaction_trigger + 1);`
` // Flush whatever is remaining in memtable. This is typically small, about`
` // 30KB.`
` ASSERT_OK(Flush(1));`
` dbfull()->TEST_WaitForCompact();`
` // Verify compaction did not happen`
`- ASSERT_EQ(NumSortedRuns(1), options.level0_file_num_compaction_trigger + 1);`
`+ ASSERT_EQ(NumSortedRuns(1), options.level0_file_num_compaction_trigger + 2);`
` ASSERT_EQ(total_picked_compactions, 0);`
` ASSERT_OK(dbfull()->SetOptions(`
`@@ -552,8 +553,8 @@ TEST_P(DBTestUniversalCompaction, DynamicUniversalCompactionReadAmplification) {`
` // If max_merge_width had not been changed dynamically above, and if it`
` // continued to be the default value of UINIT_MAX, total_picked_compactions`
` // would have been 1.`
`- ASSERT_EQ(total_picked_compactions, 2);`
`- ASSERT_EQ(total_size_ratio_compactions, 2);`
`+ ASSERT_EQ(total_picked_compactions, 3);`
`+ ASSERT_EQ(total_size_ratio_compactions, 3);`
Contributor guide
Assessment
This issue has not been assessed yet.