Bug: RepairDB pushes tables with empty smallest/largest InternalKeys causing assertion failure in Encode
- Dominant language
- C++
- Stars
- 39.4k
- Forks
- 8.2k
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
`Repairer::ScanTable` tracks whether any valid key was found via an `empty` flag but checks only the iterator status before pushing the table into `tables_`. When a corrupted SST opens with OK status but every key fails `ParseInternalKey`, the table is pushed with default-constructed (empty) `smallest`/`largest` `InternalKey`s; `WriteDescriptor` later calls `InternalKey::Encode()`, which asserts `!rep_.empty()` and aborts.
## Version
```
$ git describe --tags
1.23-91-g7ee830d
```
## Description
`ScanTable` in `db/repair.cc` scans each table to extract `smallest`/`largest`:
```cpp
// db/repair.cc:249-290 (excerpt)
bool empty = true;
ParsedInternalKey parsed;
for (iter->SeekToFirst(); iter->Valid(); iter->Next()) {
Slice key = iter->key();
if (!ParseInternalKey(key, &parsed)) {
continue; // corrupted key skipped; empty stays true
}
counter++;
if (empty) {
empty = false;
t.meta.smallest.DecodeFrom(key);
}
t.meta.largest.DecodeFrom(key);
}
if (!iter->status().ok()) {
status = iter->status();
}
// ...
if (status.ok()) {
tables_.push_back(t); // BUG: pushed even when empty == true
}
```
When the SST's data blocks are corrupted but the index block and footer remain intact, `Table::Open` succeeds and the iterator yields entries with OK status. Every key's bytes are garbage, so `ParseInternalKey` (`db/dbformat.h:171`) returns false for each — either the key is shorter than 8 bytes or the type byte exceeds `kTypeValue`. The loop never enters the body, `counter` stays 0, `empty` stays true, and `t.meta.smallest`/`largest` are never populated. Yet because `status` is OK, the table is pushed into `tables_` with invalid (empty-`rep_`) `InternalKey`s.
Later, `WriteDescriptor` serializes every table:
```cpp
// db/repair.cc:368-372
for (size_t i = 0; i < tables_.size(); i++) {
const TableInfo& t = tables_[i];
edit_.AddFile(0, t.meta.number, t.meta.file_size, t.meta.smallest,
t.meta.largest);
}
```
`VersionEdit::EncodeTo` calls `f.smallest.Encode()`, which hits:
```cpp
// db/dbformat.h:149-151
Slice Encode() const {
assert(!rep_.empty()); // CRASH
return rep_;
}
```
This is a library bug, not API misuse:
- `RepairDB` is a public `LEVELDB_EXPORT` API (`include/leveldb/db.h:157-163`) documented as the recovery tool for databases that cannot be opened — "resurrect as much of the contents as possible." Corruption is the expected input, not a forbidden one.
- `ScanTable` already maintains the `empty` flag tracking whether any valid key was found, but the push at line 288 guards on `status.ok()` alone, ignoring `empty`.
- `InternalKey`'s own constructor comment (`db/dbformat.h:139`) states an empty `rep_` means "invalid" — yet the library pushes an invalid `InternalKey` into `tables_` and later unconditionally serializes it via `Encode()`, which asserts the very invariant the library itself violated.
- The assertion checks **internal state** (`rep_` non-emptiness), not an input requirement on the caller.
In release builds (`-DNDEBUG`) the assertion is compiled out: `RepairDB` returns `Status::OK()` but writes a MANIFEST containing a new-file entry with zero-length `smallest`/`largest` keys. Reopening the repaired DB fails with `Corruption: VersionEdit: new-file entry` (the `GetInternalKey` decoder at `version_edit.cc:181` rejects the empty key). Thus `RepairDB` silently produces an unopenable database — failing its core contract with no error indication.
## PoC Code
```cpp
#include
#include
#include
#include
#include
#include
int main(){
std::string p="/tmp/ldb_poc";
system("rm -rf /tmp/ldb_poc; mkdir -p /tmp/ldb_poc");
// Create a DB, write one key, compact to produce an SST (.ldb) on disk
leveldb::Options o; o.create_if_missing=true;
o.compression=leveldb::kSnappyCompression;
leveldb::DB* d=nullptr;
if(!leveldb::DB::Open(o,p,&d).ok())return 0;
d->Put({},"a",std::string(500,'x'));
d->CompactRange(nullptr,nullptr);
delete d;
// Corrupt each .ldb: parse the footer to find where data blocks end,
// then zero only that region. The index block + footer stay intact so
// Table::Open succeeds (iterator status OK), but every data key becomes
// garbage that ParseInternalKey rejects.
DIR*dir=opendir(p.c_str());struct dirent*e;
while((e=readdir(dir))){
std::string n=e->d_name; if(n.size()<5||n.substr(n.size()-4)!=".ldb")continue;
FILE*f=fopen((p+"/"+n).c_str(),"r+b");if(!f)continue;
fseek(f,0,SEEK_END);long sz=ftell(f);fseek(f,0,SEEK_SET);
if(sz<48){fclose(f);continue;}
char*b=new char[sz];fread(b,1,sz,f);
uint8_t*ft=(uint8_t*)b+sz-48;uint64_t mo=0;int sh=0,i=0;
while(1){uint8_t by=ft[i++];mo|=uint64_t(by&0x7f)< ScanTable pushes the table with empty smallest/largest
// InternalKeys -> WriteDescriptor -> InternalKey::Encode() ->
// assert(!rep_.empty()) crashes.
leveldb::RepairDB(p,o);
}
```
## Stack Trace
```
ERROR: AddressSanitizer: ABRT on unknown address 0x000000003aea (pc 0x7e7a35247b2c bp 0x7ffe0c086960 sp 0x7ffe0c086920 T0)
#0 0x7e7a35247b2c in __pthread_kill_implementation nptl/pthread_kill.c:44:76
#1 0x7e7a35247b2c in __pthread_kill_internal nptl/pthread_kill.c:78:10
#2 0x7e7a35247b2c in pthread_kill nptl/pthread_kill.c:89:10
#3 0x7e7a351ee27d in raise signal/../sysdeps/posix/raise.c:26:13
#4 0x7e7a351d18fe in abort stdlib/abort.c:79:7
#5 0x7e7a351d181a in __assert_fail_base assert/assert.c:96:3
#6 0x7e7a351e4516 in __assert_fail assert/assert.c:105:3
#7 0x5966b9d0847a in leveldb::InternalKey::Encode() const /db/dbformat.h:150:5
#8 0x5966b9d059ec in leveldb::VersionEdit::EncodeTo(std::__cxx11::basic_string, std::allocator>*) const /db/version_edit.cc:82:44
#9 0x5966b9cf885b in leveldb::(anonymous namespace)::Repairer::WriteDescriptor() /db/repair.cc:380:13
#10 0x5966b9cf61b6 in leveldb::(anonymous namespace)::Repairer::Run() /db/repair.cc:75:16
#11 0x5966b9cf58f9 in leveldb::RepairDB(std::__cxx11::basic_string, std::allocator> const&, leveldb::Options const&) /db/repair.cc:448:19
#12 0x5966b9c8ebef in main /poc.cpp:43:3
#13 0x7e7a351d31c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
#14 0x7e7a351d328a in __libc_start_main csu/../csu/libc-start.c:360:3
#15 0x5966b9ba28e4 in _start (/poc+0xe38e4) (BuildId: 1574ecc6605c554f613b342f73c70db3962a6d39)
Register values:
rax = 0x0000000000000000 rbx = 0x0000000000003aea rcx = 0x00007e7a35247b2c rdx = 0x0000000000000006
rdi = 0x0000000000003aea rsi = 0x0000000000003aea rbp = 0x00007ffe0c086960 rsp = 0x00007ffe0c086920
r8 = 0x0000000000000085 r9 = 0x00007cba345e0000 r10 = 0x0000000000000008 r11 = 0x0000000000000246
r12 = 0x0000000000000006 r13 = 0x00005966b9db26cf r14 = 0x0000000000000016 r15 = 0x00005966b9dd5a20
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: ABRT nptl/pthread_kill.c:44:76 in __pthread_kill_implementation
```
## Reproduction Step
```bash
# 1. Checkout leveldb at the version above
git clone https://github.com/google/leveldb.git
cd leveldb
git checkout 7ee830d02b623e8ffe0b95d59a74db1e58da04c5
# 2. Build the library (sanitizer flags)
export CC=clang CXX=clang++
export CFLAGS="-fsanitize=address,undefined -g -O0 -fPIC -fno-omit-frame-pointer"
export CXXFLAGS="$CFLAGS"
export WORK=$(pwd)/build/sanitizer
mkdir -p build && cd build
cmake -DCMAKE_INSTALL_PREFIX="$WORK" \
-DCMAKE_C_COMPILER="$CC" -DCMAKE_CXX_COMPILER="$CXX" \
-DCMAKE_C_FLAGS="$CFLAGS" -DCMAKE_CXX_FLAGS="$CXXFLAGS" \
-DBUILD_SHARED_LIBS=OFF \
-DLEVELDB_BUILD_TESTS=OFF \
-DLEVELDB_BUILD_BENCHMARKS=OFF \
-DLEVELDB_INSTALL=ON \
..
make -j$(nproc) && make install
# 3. Build the PoC against that build
clang++ -g -O0 -fsanitize=address,undefined \
-I"$WORK/include" \
poc.cpp -o poc \
-Wl,--start-group $WORK/lib/lib*.a -Wl,--end-group \
-lsnappy
# 4. Run
./poc
```
## Submission Statement
This report was produced by FuzzAnything's AI-assisted library fuzzer and manually verified by a team member. We reviewed the PoC against the upstream API documentation — call order, parameters, and memory ownership — and found no API misuse.
> Signed-off-by: FuzzAnything
Contributor guide
Research direction
Start with ScanTable and WriteDescriptor in db/repair.cc, then inspect InternalKey::Encode in db/dbformat.h and the VersionEdit serialization path. Run the provided corrupted-SST PoC under sanitizers and verify that RepairDB no longer asserts or produces an unopenable database when no valid internal keys are found.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 62/100