DataBlockHashIndex does not validate number of buckets upon read
- Dominant language
- C++
- Stars
- 32.1k
- Forks
- 6.9k
- Avg merge
- 32m
- Merged PRs (30d)
- 1
Description
rocksdb version:6.5.2
After the server is powered off and restarted, a crash occurred during database recovery, here is process statck:
code:
https://github.com/facebook/rocksdb/blob/f48aa1c3084700bc72b73ee36f027e428f0dda86/table/block_based/data_block_hash_index.cc#L75-L83
stack:
```bash
gdb) bt
#0 0x00007f433f087428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1 0x00007f433f08902a in __GI_abort () at abort.c:89
#2 0x00007f433f07fbd7 in __assert_fail_base (fmt=, assertion=assertion@entry=0x7f4340de7ae8 "size > num_buckets_ * sizeof(uint8_t)", file=file@entry=0x7f4340de7a38 "/home/xxp/work/rocksdb-6.5.2/table/block_based/data_block_hash_index.cc", line=line@entry=80, function=function@entry=0x7f4340de7c00 "void rocksdb::DataBlockHashIndex::Initialize(const char*, uint16_t, uint16_t*)") at assert.c:92
#3 0x00007f433f07fc82 in __GI___assert_fail (assertion=0x7f4340de7ae8 "size > num_buckets_ * sizeof(uint8_t)", file=0x7f4340de7a38 "/home/xxp/work/rocksdb-6.5.2/table/block_based/data_block_hash_index.cc", line=80, function=0x7f4340de7c00 "void rocksdb::DataBlockHashIndex::Initialize(const char*, uint16_t, uint16_t*)") at assert.c:101
#4 0x00007f4340b83c5c in rocksdb::DataBlockHashIndex::Initialize (this=0x7fff9ac0d5e0, data=0x22606c0 "a0-31d8-46a7-83cf-4ba066aada2d\022\204\003\002\376\006\353\374\020\002\v\370\365\377\375\360\f\370\361", , size=91, map_offset=0x7fff9ac0d43e) at /home/xxp/work/rocksdb-6.5.2/table/block_based/data_block_hash_index.cc:80
#5 0x00007f4340b37c3e in rocksdb::Block::Block (this=0x7fff9ac0d590, contents=..., _global_seqno=18446744073709551615, read_amp_bytes_per_bit=0, statistics=0x0) at /home/xxp/work/rocksdb-6.5.2/table/block_based/block.cc:864
#6 0x00007f4340bb2876 in rocksdb::ReadTableProperties (file=0x22a68d0, file_size=67565882, table_magic_number=0, ioptions=..., properties=0x7fff9ac0edf8, compression_type_missing=false, memory_allocator=0x0) at /home/xxp/work/rocksdb-6.5.2/table/meta_blocks.cc:388
#7 0x00007f4340a25380 in rocksdb::Version::GetTableProperties (this=0x2260a50, tp=0x7fff9ac0eef0 = {...}, file_meta=0x227cf90, fname=0x0) at /home/xxp/work/rocksdb-6.5.2/db/version_set.cc:1234
#8 0x00007f4340a2acda in rocksdb::Version::MaybeInitializeFileMetaData (this=0x2260a50, file_meta=0x227cf90) at /home/xxp/work/rocksdb-6.5.2/db/version_set.cc:2046
#9 0x00007f4340a2b179 in rocksdb::Version::UpdateAccumulatedStats (this=0x2260a50, update_stats=true) at /home/xxp/work/rocksdb-6.5.2/db/version_set.cc:2107
#10 0x00007f4340a2abbd in rocksdb::Version::PrepareApply (this=0x2260a50, mutable_cf_options=..., update_stats=true) at /home/xxp/work/rocksdb-6.5.2/db/version_set.cc:2030
#11 0x00007f4340a38109 in rocksdb::VersionSet::Recover (this=0x224a180, column_families=std::vector of length 2, capacity 2 = {...}, read_only=false, db_id=0x2243af0) at /home/xxp/work/rocksdb-6.5.2/db/version_set.cc:4448
#12 0x00007f434094f01c in rocksdb::DBImpl::Recover (this=0x2243ac0, column_families=std::vector of length 2, capacity 2 = {...}, read_only=false, error_if_log_file_exist=false, error_if_data_exists_in_logs=false) at /home/xxp/work/rocksdb-6.5.2/db/db_impl/db_impl_open.cc:396
#13 0x00007f4340954d23 in rocksdb::DBImpl::Open (db_options=..., dbname="/home/xxp/work/repo", column_families=std::vector of length 2, capacity 2 = {...}, handles=0x7fff9ac109c0, dbptr=0x626340 , seq_per_batch=false, batch_per_txn=true) at /home/xxp/work/rocksdb-6.5.2/db/db_impl/db_impl_open.cc:1358
#14 0x00007f434095416b in rocksdb::DB::Open (db_options=..., dbname="/home/xxp/work/repo", column_families=std::vector of length 2, capacity 2 = {...}, handles=0x7fff9ac109c0, dbptr=0x626340 ) at /home/xxp/work/rocksdb-6.5.2/db/db_impl/db_impl_open.cc:1251
#15 0x00000000004155b8 in Storage::init (this=0x626320 , db_dir="/home/xxp/work/repo", mode=Storage::OPEN_MODE_READ_WRITE, db_dir2="") at /home/xxp/work/rocksdb-6.5.2/demo/Storage.cpp:119
#16 0x000000000041e2ce in main (argc=1, argv=0x7fff9ac11858) at /home/xxp/work/rocksdb-6.5.2/demo/main.cpp:9
#17 0x00007f433f072830 in __libc_start_main (main=0x41e262 , argc=1, argv=0x7fff9ac11858, init=, fini=, rtld_fini=, stack_end=0x7fff9ac11848) at ../csu/libc-start.c:291
#18 0x0000000000414d19 in _start ()
```
and here is some memory value:
```bash
gdb) p size
$6 = 91
(gdb) p num_buckets_
$7 = 65024
(gdb) x/91x data
0x22606c0: 0x61 0x30 0x2d 0x33 0x31 0x64 0x38 0x2d
0x22606c8: 0x34 0x36 0x61 0x37 0x2d 0x38 0x33 0x63
0x22606d0: 0x66 0x2d 0x34 0x62 0x61 0x30 0x36 0x36
0x22606d8: 0x61 0x61 0x64 0x61 0x32 0x64 0x12 0x84
0x22606e0: 0x03 0x02 0xfe 0x06 0xeb 0xfc 0x10 0x02
0x22606e8: 0x0b 0xf8 0xf5 0xff 0xfd 0xf0 0x0c 0xf8
0x22606f0: 0xf1 0xf5 0x00 0x0a 0xfc 0x01 0x0a 0xff
0x22606f8: 0x0c 0x09 0x00 0x0c 0x15 0x03 0x10 0x06
0x2260700: 0xeb 0xf5 0x00 0xf7 0x02 0x0b 0x0a 0x04
0x2260708: 0x10 0x00 0xff 0x00 0xfa 0x08 0x0c 0xfb
0x2260710: 0x03 0xfb 0x11 0xec 0x05 0xf6 0x06 0xf4
0x2260718: 0x00 0x00 0xfe
(gdb) x/91c data
0x22606c0: 97 'a' 48 '0' 45 '-' 51 '3' 49 '1' 100 'd' 56 '8' 45 '-'
0x22606c8: 52 '4' 54 '6' 97 'a' 55 '7' 45 '-' 56 '8' 51 '3' 99 'c'
0x22606d0: 102 'f' 45 '-' 52 '4' 98 'b' 97 'a' 48 '0' 54 '6' 54 '6'
0x22606d8: 97 'a' 97 'a' 100 'd' 97 'a' 50 '2' 100 'd' 18 '\022' -124 '\204'
0x22606e0: 3 '\003' 2 '\002' -2 '\376' 6 '\006' -21 '\353' -4 '\374' 16 '\020' 2 '\002'
0x22606e8: 11 '\v' -8 '\370' -11 '\365' -1 '\377' -3 '\375' -16 '\360' 12 '\f' -8 '\370'
0x22606f0: -15 '\361' -11 '\365' 0 '\000' 10 '\n' -4 '\374' 1 '\001' 10 '\n' -1 '\377'
0x22606f8: 12 '\f' 9 '\t' 0 '\000' 12 '\f' 21 '\025' 3 '\003' 16 '\020' 6 '\006'
0x2260700: -21 '\353' -11 '\365' 0 '\000' -9 '\367' 2 '\002' 11 '\v' 10 '\n' 4 '\004'
0x2260708: 16 '\020' 0 '\000' -1 '\377' 0 '\000' -6 '\372' 8 '\b' 12 '\f' -5 '\373'
0x2260710: 3 '\003' -5 '\373' 17 '\021' -20 '\354' 5 '\005' -10 '\366' 6 '\006' -12 '\364'
0x2260718: 0 '\000' 0 '\000' -2 '\376'
```
Contributor guide
Assessment
This issue has not been assessed yet.