DB::DeleteFile vs DB::CompactRange race
- Dominant language
- C++
- Stars
- 32.1k
- Forks
- 6.9k
- Avg merge
- 32m
- Merged PRs (30d)
- 1
Description
### Expected behavior
DeleteFile fails, Manual compaction completed successfully, after that writes into RocksDB and flush succeeds.
### Actual behavior
Manual compaction fails with "Corruption: Compaction input files inconsistent", subsequent writes and flush fail with the same error:
```
./deletefile_test --gtest_filter=DeleteFileTest.DeleteWithManualCompaction
Note: Google Test filter = DeleteFileTest.DeleteWithManualCompaction
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from DeleteFileTest
[ RUN ] DeleteFileTest.DeleteWithManualCompaction
db/deletefile_test.cc:583: Failure
manual_compaction_status
Corruption: Compaction input files inconsistent
db/deletefile_test.cc:62: Failure
db_->Put(options, key, value)
Corruption: Compaction input files inconsistent
db/deletefile_test.cc:586: Failure
dbi->TEST_FlushMemTable( true)
Corruption: Compaction input files inconsistent
[ FAILED ] DeleteFileTest.DeleteWithManualCompaction (1517 ms)
```
### Steps to reproduce the behavior
1) `git checkout v6.20.3`
2) `git apply` the following patch:
```
diff --git a/db/deletefile_test.cc b/db/deletefile_test.cc
index 38bc49602..09ee630bd 100644
--- a/db/deletefile_test.cc
+++ b/db/deletefile_test.cc
@@ -551,6 +551,43 @@ TEST_F(DeleteFileTest, DeleteNonDefaultColumnFamily) {
}
}
+TEST_F(DeleteFileTest, DeleteWithManualCompaction) {
+ constexpr auto kNumSst = 3;
+ constexpr auto kNumKeysPerSst = 10000;
+
+ DBImpl* dbi = reinterpret_cast(db_);
+ for (auto num = 0; num < kNumSst; num++) {
+ AddKeys(kNumKeysPerSst, 0);
+ ASSERT_OK(dbi->TEST_FlushMemTable(/* wait = */ true));
+ }
+
+ std::vector metadata;
+ db_->GetLiveFilesMetaData(&metadata);
+
+ std::atomic manual_compaction_done{false};
+ Status manual_compaction_status;
+
+ std::thread manual_compaction([this, &manual_compaction_done, &manual_compaction_status] {
+ manual_compaction_status = db_->CompactRange(
+ rocksdb::CompactRangeOptions(), /* begin = */ nullptr, /* end = */ nullptr);
+ manual_compaction_done = true;
+ });
+
+ while (!manual_compaction_done) {
+ for (auto& file_meta : metadata) {
+ db_->DeleteFile(file_meta.name);
+ }
+ }
+
+ manual_compaction.join();
+ EXPECT_OK(manual_compaction_status);
+
+ AddKeys(10, 0);
+ ASSERT_OK(dbi->TEST_FlushMemTable(/* wait = */ true));
+
+ Close();
+}
+
} // namespace ROCKSDB_NAMESPACE
#ifdef ROCKSDB_UNITTESTS_WITH_CUSTOM_OBJECTS_FROM_STATIC_LIBS
```
3) `./deletefile_test --gtest_filter=DeleteFileTest.DeleteWithManualCompaction`
Contributor guide
Assessment
This issue has not been assessed yet.