facebook / facebook/rocksdb

jemalloc_nodump_allocator should not call malloc_usable_size on memory allocated from arenas

Open
#10,797 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

### Expected behavior
ASAN doesn't complain

### Actual behavior
ASAN complains with something like
```
==661==ERROR: AddressSanitizer: attempting to call malloc_usable_size() for pointer which is not owned: 0x7f6a99501000
#0 0x38b3ed in malloc_usable_size (/opt/rockset/rs/cpp/out/debug-clang-sanitize-address/leaf/leaf_server+0x38b3ed) (BuildId: 059497e5183f7676)
#1 0x30b57f in __asan::asan_malloc_usable_size(void const*, unsigned long, unsigned long) crtstuff.c
#2 0x7f6b19625b6c in rocksdb::BlockContents::usable_size() const /src/rocksdb/./table/format.h:294:37
#3 0x7f6b19625b6c in rocksdb::Block::usable_size() const /src/rocksdb/./table/block_based/block.h:166:60
#4 0x7f6b19625b6c in rocksdb::Block::ApproximateMemoryUsage() const /src/rocksdb/table/block_based/block.cc:1117:29
#5 0x7f6b19639c5c in rocksdb::Status rocksdb::BlockBasedTableBuilder::InsertBlockInCache(rocksdb::Slice const&, rocksdb::BlockHandle const*, rocksdb::BlockType) /src/rocksdb/table/block_based/block_based_table_builder.cc:1545:57
#6 0x7f6b1963199e in rocksdb::BlockBasedTableBuilder::InsertBlockInCacheHelper(rocksdb::Slice const&, rocksdb::BlockHandle const*, rocksdb::BlockType, bool) /src/rocksdb/table/block_based/block_based_table_builder.cc:1497:34
#7 0x7f6b19632326 in rocksdb::BlockBasedTableBuilder::WriteRawBlock(rocksdb::Slice const&, rocksdb::CompressionType, rocksdb::BlockHandle*, rocksdb::BlockType, rocksdb::Slice const*, bool) /src/rocksdb/table/block_based/block_based_table_builder.cc:1314:37
#8 0x7f6b19632bd9 in rocksdb::BlockBasedTableBuilder::WriteBlock(rocksdb::Slice const&, rocksdb::BlockHandle*, rocksdb::BlockType) /src/rocksdb/table/block_based/block_based_table_builder.cc:1094:16
#9 0x7f6b1963534b in rocksdb::BlockBasedTableBuilder::EnterUnbuffered() /src/rocksdb/table/block_based/block_based_table_builder.cc:1975:17
#10 0x7f6b19635e57 in rocksdb::BlockBasedTableBuilder::Finish() /src/rocksdb/table/block_based/block_based_table_builder.cc:2007:20
#11 0x7f6b1933c6be in rocksdb::BuildTable(std::__cxx11::basic_string, std::allocator > const&, rocksdb::VersionSet*, rocksdb::ImmutableDBOptions const&, rocksdb::TableBuilderOptions const&, rocksdb::FileOptions const&, rocksdb::TableCache*, rocksdb::InternalIteratorBase*, std::vector >, std::allocator > > >, rocksdb::FileMetaData*, std::vector >*, std::vector >, unsigned long, unsigned long, rocksdb::SnapshotChecker*, bool, rocksdb::InternalStats*, rocksdb::IOStatus*, std::shared_ptr const&, rocksdb::BlobFileCreationReason, rocksdb::EventLogger*, int, rocksdb::Env::IOPriority, rocksdb::TableProperties*, rocksdb::Env::WriteLifeTimeHint, std::__cxx11::basic_string, std::allocator > const*, rocksdb::BlobFileCompletionCallback*, unsigned long*, unsigned long*, unsigned long*) /src/rocksdb/db/builder.cc:261:26
#12 0x7f6b19479cd2 in rocksdb::FlushJob::WriteLevel0Table() /src/rocksdb/db/flush_job.cc:930:21
#13 0x7f6b1947c14e in rocksdb::FlushJob::Run(rocksdb::LogsWithPrepTracker*, rocksdb::FileMetaData*, bool*) /src/rocksdb/db/flush_job.cc:279:25
#14 0x7f6b193f8837 in rocksdb::DBImpl::AtomicFlushMemTablesToOutputFiles(rocksdb::autovector const&, bool*, rocksdb::JobContext*, rocksdb::LogBuffer*, rocksdb::Env::Priority) /src/rocksdb/db/db_impl/db_impl_compaction_flush.cc:527:41
#15 0x7f6b193f9f82 in rocksdb::DBImpl::FlushMemTablesToOutputFiles(rocksdb::autovector const&, bool*, rocksdb::JobContext*, rocksdb::LogBuffer*, rocksdb::Env::Priority) /src/rocksdb/db/db_impl/db_impl_compaction_flush.cc:364:74
#16 0x7f6b193fb5fa in rocksdb::DBImpl::BackgroundFlush(bool*, rocksdb::JobContext*, rocksdb::LogBuffer*, rocksdb::FlushReason*, rocksdb::Env::Priority) /src/rocksdb/db/db_impl/db_impl_compaction_flush.cc:2807:41
#17 0x7f6b194004a0 in rocksdb::DBImpl::BackgroundCallFlush(rocksdb::Env::Priority) /src/rocksdb/db/db_impl/db_impl_compaction_flush.cc:2847:51
```

### Steps to reproduce the behavior
Build with ASAN, do a background flush

The fix would be to use `nallocx(p, flags)` instead of `malloc_usable_size(p)`, with the same flags as passed to the `mallocx` call (`MALLOCX_ARENA(arena_index_) | GetThreadSpecificCache(size)`)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.