RocksDB throws segfault during flush when prefix_extractor is null
- Dominant language
- C++
- Stars
- 32.1k
- Forks
- 6.9k
- Avg merge
- 32m
- Merged PRs (30d)
- 1
Description
RocksDB version - 8.3.2
Java version - openJDK 11 or openJDK 17
I have configured the index type of block based table as `kHashSearch`
However, the prefix_extractor was not configured in the column family options (although configured in base options)
This led to a weird issue where all reads and writes were working fine for RocksDB until the moment a flush was triggered.
IMO, if prefix_extractor is a strict requirement for `kHashSearch` to work, **an error should either be thrown when the column family is created** OR **an error when we do the first write** OR some default implementation of extractor should be used. Such failures which only occur after some amount of data is written make it hard to capture scenarios in tests.
```
Stack: [0x00007f9ff11e6000,0x00007f9ff1be5000], sp=0x00007f9ff1bdec20, free space=10211k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [librocksdbjni13498244389108629237.so+0x6c0bb7] rocksdb::HashIndexBuilder::OnKeyAdded(rocksdb::Slice const&)+0x47
C [librocksdbjni13498244389108629237.so+0x66c3a0] rocksdb::BlockBasedTableBuilder::Add(rocksdb::Slice const&, rocksdb::Slice const&)+0x1d0
C [librocksdbjni13498244389108629237.so+0x3493f8] rocksdb::BuildTable(std::string const&, rocksdb::VersionSet*, rocksdb::ImmutableDBOptions const&, rocksdb::TableBuilderOptions const&, rocksdb::FileOptions co
nst&, rocksdb::ReadOptions const&, rocksdb::TableCache*, rocksdb::InternalIteratorBase*, std::vector >, std::allocator > > >, rocksdb::FileMetaData*, std::vecto
r >*, std::vector >, unsigned long, unsigned long, rocksdb::SnapshotChecker*, bool, rocksdb::Int
ernalStats*, rocksdb::IOStatus*, std::shared_ptr const&, rocksdb::BlobFileCreationReason, rocksdb::SeqnoToTimeMapping const&, rocksdb::EventLogger*, int, rocksdb::Env::IOPriority, rocksdb::Ta
bleProperties*, rocksdb::Env::WriteLifeTimeHint, std::string const*, rocksdb::BlobFileCompletionCallback*, rocksdb::Version*, unsigned long*, unsigned long*, unsigned long*)+0x12b8
C [librocksdbjni13498244389108629237.so+0x4973af] rocksdb::FlushJob::WriteLevel0Table()+0xf0f
C [librocksdbjni13498244389108629237.so+0x499142] rocksdb::FlushJob::Run(rocksdb::LogsWithPrepTracker*, rocksdb::FileMetaData*, bool*)+0x732
C [librocksdbjni13498244389108629237.so+0x415eda] rocksdb::DBImpl::FlushMemTableToOutputFile(rocksdb::ColumnFamilyData*, rocksdb::MutableCFOptions const&, bool*, rocksdb::JobContext*, rocksdb::FlushReason, rocksdb::SuperVersionContext*, std::vector >&, unsigned long, rocksdb::SnapshotChecker*, rocksdb::LogBuffer*, rocksdb::Env::Priority)+0x96a
C [librocksdbjni13498244389108629237.so+0x41982f] rocksdb::DBImpl::FlushMemTablesToOutputFiles(rocksdb::autovector const&, bool*, rocksdb::JobContext*, rocksdb::LogBuffer*, rocksdb::Env::Priority)+0xff
C [librocksdbjni13498244389108629237.so+0x41a74c] rocksdb::DBImpl::BackgroundFlush(bool*, rocksdb::JobContext*, rocksdb::LogBuffer*, rocksdb::FlushReason*, rocksdb::Env::Priority)+0xe6c
C [librocksdbjni13498244389108629237.so+0x41dc08] rocksdb::DBImpl::BackgroundCallFlush(rocksdb::Env::Priority)+0xc8
C [librocksdbjni13498244389108629237.so+0x775bcb] rocksdb::ThreadPoolImpl::Impl::BGThread(unsigned long)+0x24b
C [librocksdbjni13498244389108629237.so+0x775da2] rocksdb::ThreadPoolImpl::Impl::BGThreadWrapper(void*)+0x62
```
Contributor guide
Research direction
Start at rocksdb::HashIndexBuilder::OnKeyAdded in the flush stack, then trace how the null prefix_extractor reaches BlockBasedTableBuilder::Add when kHashSearch is configured. Reproduce the Java configuration with the column-family extractor unset and trigger a flush; done means the invalid configuration is rejected before flush or the flush no longer segfaults.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100