facebook / facebook/rocksdb

RocksDB throws segfault during flush when prefix_extractor is null

Open
#12,032 1 comment 0 reactions 0 assignees View on GitHub
bug up-for-grabs
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

RocksDB version - 8.3.2
Java version - openJDK 11 or openJDK 17

I have configured the index type of block based table as `kHashSearch`
However, the prefix_extractor was not configured in the column family options (although configured in base options)

This led to a weird issue where all reads and writes were working fine for RocksDB until the moment a flush was triggered.

IMO, if prefix_extractor is a strict requirement for `kHashSearch` to work, **an error should either be thrown when the column family is created** OR **an error when we do the first write** OR some default implementation of extractor should be used. Such failures which only occur after some amount of data is written make it hard to capture scenarios in tests.

```
Stack: [0x00007f9ff11e6000,0x00007f9ff1be5000], sp=0x00007f9ff1bdec20, free space=10211k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [librocksdbjni13498244389108629237.so+0x6c0bb7] rocksdb::HashIndexBuilder::OnKeyAdded(rocksdb::Slice const&)+0x47
C [librocksdbjni13498244389108629237.so+0x66c3a0] rocksdb::BlockBasedTableBuilder::Add(rocksdb::Slice const&, rocksdb::Slice const&)+0x1d0
C [librocksdbjni13498244389108629237.so+0x3493f8] rocksdb::BuildTable(std::string const&, rocksdb::VersionSet*, rocksdb::ImmutableDBOptions const&, rocksdb::TableBuilderOptions const&, rocksdb::FileOptions co
nst&, rocksdb::ReadOptions const&, rocksdb::TableCache*, rocksdb::InternalIteratorBase*, std::vector >, std::allocator > > >, rocksdb::FileMetaData*, std::vecto
r >*, std::vector >, unsigned long, unsigned long, rocksdb::SnapshotChecker*, bool, rocksdb::Int
ernalStats*, rocksdb::IOStatus*, std::shared_ptr const&, rocksdb::BlobFileCreationReason, rocksdb::SeqnoToTimeMapping const&, rocksdb::EventLogger*, int, rocksdb::Env::IOPriority, rocksdb::Ta
bleProperties*, rocksdb::Env::WriteLifeTimeHint, std::string const*, rocksdb::BlobFileCompletionCallback*, rocksdb::Version*, unsigned long*, unsigned long*, unsigned long*)+0x12b8
C [librocksdbjni13498244389108629237.so+0x4973af] rocksdb::FlushJob::WriteLevel0Table()+0xf0f
C [librocksdbjni13498244389108629237.so+0x499142] rocksdb::FlushJob::Run(rocksdb::LogsWithPrepTracker*, rocksdb::FileMetaData*, bool*)+0x732
C [librocksdbjni13498244389108629237.so+0x415eda] rocksdb::DBImpl::FlushMemTableToOutputFile(rocksdb::ColumnFamilyData*, rocksdb::MutableCFOptions const&, bool*, rocksdb::JobContext*, rocksdb::FlushReason, rocksdb::SuperVersionContext*, std::vector >&, unsigned long, rocksdb::SnapshotChecker*, rocksdb::LogBuffer*, rocksdb::Env::Priority)+0x96a
C [librocksdbjni13498244389108629237.so+0x41982f] rocksdb::DBImpl::FlushMemTablesToOutputFiles(rocksdb::autovector const&, bool*, rocksdb::JobContext*, rocksdb::LogBuffer*, rocksdb::Env::Priority)+0xff
C [librocksdbjni13498244389108629237.so+0x41a74c] rocksdb::DBImpl::BackgroundFlush(bool*, rocksdb::JobContext*, rocksdb::LogBuffer*, rocksdb::FlushReason*, rocksdb::Env::Priority)+0xe6c
C [librocksdbjni13498244389108629237.so+0x41dc08] rocksdb::DBImpl::BackgroundCallFlush(rocksdb::Env::Priority)+0xc8
C [librocksdbjni13498244389108629237.so+0x775bcb] rocksdb::ThreadPoolImpl::Impl::BGThread(unsigned long)+0x24b
C [librocksdbjni13498244389108629237.so+0x775da2] rocksdb::ThreadPoolImpl::Impl::BGThreadWrapper(void*)+0x62
```

Contributor guide

Open the contributing guide

Research direction

Start at rocksdb::HashIndexBuilder::OnKeyAdded in the flush stack, then trace how the null prefix_extractor reaches BlockBasedTableBuilder::Add when kHashSearch is configured. Reproduce the Java configuration with the column-family extractor unset and trigger a flush; done means the invalid configuration is rejected before flush or the flush no longer segfaults.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.