facebook / facebook/rocksdb

~LocalSavePoint() { assert(committed_); }

Open
#8,080 13 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

### Steps to reproduce the behavior
[ClickHouse](https://github.com/ClickHouse/ClickHouse/) supports tables that are stored in rocksdb.
We run stress tests and got assertion in rocksdb code:

https://github.com/ClickHouse-Extras/rocksdb/blob/fdf882ded218344c136c97daf76dfb59e4bc155f/db/write_batch_internal.h#L226
``` c++
#ifndef NDEBUG
~LocalSavePoint() { assert(committed_); }
#endif
```

```sql
CREATE TABLE test (key UInt64, value String) Engine=EmbeddedRocksDB PRIMARY KEY(key);
INSERT INTO test SELECT number % 77 AS k, SUM(number) AS value, (1, 1.2), bitmapBuild(groupArray(number)) FROM numbers(10000000) group by k;
```

stack trace
```
2021.03.05 12:47:57.622821 [ 386235 ] {} BaseDaemon: ########################################
2021.03.05 12:47:57.623345 [ 386235 ] {} BaseDaemon: (version 21.3.1.6178 (official build), build id: 05382CBC774A3BA42435A5C322CEF9474DC1347B) (from thread 3210) (query_id: 918637a4-825b-4fe0-8696-79409e160138) Received signal Aborted (6)
2021.03.05 12:47:57.623714 [ 386235 ] {} BaseDaemon:
2021.03.05 12:47:57.624130 [ 386235 ] {} BaseDaemon: Stack trace: 0x7f9022d5718b 0x7f9022d36859 0x7f9022d36729 0x7f9022d47f36 0x20979697 0x20975789 0x209759a5 0x1abc9b51 0x1abc95d8 0x19e94c88 0x19e92fd2 0x19e9072e 0x19e9082c 0x19d2fa67 0x1b3ee687 0x1b0818fb 0x194f2201 0x1b0df456 0x1b0df2c1 0x1b0df88b 0x1b0df2c1 0x1b0df88b 0x1b0df2c1 0x1b0df88b 0x1b0df2c1 0x1b0df958 0x1b0df2c1 0x1b0df958 0x1b0df2c1 0x1b0df958 0x1b0e277e 0x1b0e2f99 0x1b0e0ea4 0x1b0e01a6 0x1b0fc3b7 0x1b0fc2f6 0x1b0fc27d 0x1b0fc241 0x1b0fc182 0x1b0fc08c 0x1b0fbfbd 0x1b0fbf8d 0x1b0fbf65 0x1b0fbf40 0x10fc09d9 0x10fbfcf5 0x10fe326e 0x10fea5d4 0x10fea55d 0x10fea4b5 0x10fe9e82 0x7f9022f0c609
2021.03.05 12:47:57.624600 [ 386235 ] {} BaseDaemon: 4. gsignal @ 0x4618b in /usr/lib/x86_64-linux-gnu/libc-2.31.so
2021.03.05 12:47:57.624961 [ 386235 ] {} BaseDaemon: 5. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
2021.03.05 12:47:57.625139 [ 386235 ] {} BaseDaemon: 6. ? @ 0x25729 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
2021.03.05 12:47:57.625539 [ 386235 ] {} BaseDaemon: 7. ? @ 0x36f36 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
2021.03.05 12:47:58.697576 [ 386235 ] {} BaseDaemon: 8. ./obj-x86_64-linux-gnu/../contrib/rocksdb/db/write_batch_internal.h:226: rocksdb::LocalSavePoint::~LocalSavePoint() @ 0x20979697 in /usr/bin/clickhouse
2021.03.05 12:47:59.646143 [ 386235 ] {} BaseDaemon: 9. ./obj-x86_64-linux-gnu/../contrib/rocksdb/db/write_batch.cc:793: rocksdb::WriteBatchInternal::Put(rocksdb::WriteBatch*, unsigned int, rocksdb::Slice const&, rocksdb::Slice const&) @ 0x20975789 in /usr/bin/clickhouse
2021.03.05 12:48:00.417626 [ 386235 ] {} BaseDaemon: 10. ./obj-x86_64-linux-gnu/../contrib/rocksdb/db/write_batch.cc:797: rocksdb::WriteBatch::Put(rocksdb::ColumnFamilyHandle*, rocksdb::Slice const&, rocksdb::Slice const&) @ 0x209759a5 in /usr/bin/clickhouse
2021.03.05 12:48:00.839316 [ 386235 ] {} BaseDaemon: 11. ./obj-x86_64-linux-gnu/../contrib/rocksdb/include/rocksdb/write_batch.h:72: rocksdb::WriteBatch::Put(rocksdb::Slice const&, rocksdb::Slice const&) @ 0x1abc9b51 in /usr/bin/clickhouse
2021.03.05 12:48:01.250738 [ 386235 ] {} BaseDaemon: 12. ./obj-x86_64-linux-gnu/../src/Storages/RocksDB/EmbeddedRocksDBBlockOutputStream.cpp:57: DB::EmbeddedRocksDBBlockOutputStream::write(DB::Block const&) @ 0x1abc95d8 in /usr/bin/clickhouse
2021.03.05 12:48:02.768004 [ 386235 ] {} BaseDaemon: 13. ./obj-x86_64-linux-gnu/../src/DataStreams/PushingToViewsBlockOutputStream.cpp:0: DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0x19e94c88 in /usr/bin/clickhouse
2021.03.05 12:48:03.095648 [ 386235 ] {} BaseDaemon: 14. ./obj-x86_64-linux-gnu/../src/DataStreams/AddingDefaultBlockOutputStream.cpp:24: DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0x19e92fd2 in /usr/bin/clickhouse
2021.03.05 12:48:03.273878 [ 386235 ] {} BaseDaemon: 15. ./obj-x86_64-linux-gnu/../src/DataStreams/SquashingBlockOutputStream.cpp:30: DB::SquashingBlockOutputStream::finalize() @ 0x19e9072e in /usr/bin/clickhouse
2021.03.05 12:48:03.423804 [ 386235 ] {} BaseDaemon: 16. ./obj-x86_64-linux-gnu/../src/DataStreams/SquashingBlockOutputStream.cpp:50: DB::SquashingBlockOutputStream::writeSuffix() @ 0x19e9082c in /usr/bin/clickhouse
2021.03.05 12:48:03.616951 [ 386235 ] {} BaseDaemon: 17. ./obj-x86_64-linux-gnu/../src/DataStreams/CountingBlockOutputStream.h:37: DB::CountingBlockOutputStream::writeSuffix() @ 0x19d2fa67 in /usr/bin/clickhouse
2021.03.05 12:48:03.796141 [ 386235 ] {} BaseDaemon: 18. ./obj-x86_64-linux-gnu/../src/Processors/Sources/SinkToOutputStream.cpp:23: DB::SinkToOutputStream::onFinish() @ 0x1b3ee687 in /usr/bin/clickhouse
2021.03.05 12:48:03.993202 [ 386235 ] {} BaseDaemon: 19. ./obj-x86_64-linux-gnu/../src/Processors/ISink.cpp:20: DB::ISink::prepare() @ 0x1b0818fb in /usr/bin/clickhouse
2021.03.05 12:48:04.500659 [ 386235 ] {} BaseDaemon: 20. ./obj-x86_64-linux-gnu/../src/Processors/IProcessor.h:186: DB::IProcessor::prepare(std::__1::vector > const&, std::__1::vector > const&) @ 0x194f2201 in /usr/bin/clickhouse
2021.03.05 12:48:05.252061 [ 386235 ] {} BaseDaemon: 21. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:215: DB::PipelineExecutor::prepareProcessor(unsigned long, unsigned long, std::__1::queue > >&, std::__1::queue > >&, std::__1::unique_lock) @ 0x1b0df456 in /usr/bin/clickhouse
2021.03.05 12:48:05.970880 [ 386235 ] {} BaseDaemon: 22. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:188: DB::PipelineExecutor::tryAddProcessorToStackIfUpdated(DB::ExecutingGraph::Edge&, std::__1::queue > >&, std::__1::queue > >&, unsigned long) @ 0x1b0df2c1 in /usr/bin/clickhouse
2021.03.05 12:48:06.801972 [ 386235 ] {} BaseDaemon: 23. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:285: DB::PipelineExecutor::prepareProcessor(unsigned long, unsigned long, std::__1::queue > >&, std::__1::queue > >&, std::__1::unique_lock) @ 0x1b0df88b in /usr/bin/clickhouse
2021.03.05 12:48:07.605719 [ 386235 ] {} BaseDaemon: 24. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:188: DB::PipelineExecutor::tryAddProcessorToStackIfUpdated(DB::ExecutingGraph::Edge&, std::__1::queue > >&, std::__1::queue > >&, unsigned long) @ 0x1b0df2c1 in /usr/bin/clickhouse
2021.03.05 12:48:08.432691 [ 386235 ] {} BaseDaemon: 25. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:285: DB::PipelineExecutor::prepareProcessor(unsigned long, unsigned long, std::__1::queue > >&, std::__1::queue > >&, std::__1::unique_lock) @ 0x1b0df88b in /usr/bin/clickhouse
2021.03.05 12:48:09.105593 [ 386235 ] {} BaseDaemon: 26. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:188: DB::PipelineExecutor::tryAddProcessorToStackIfUpdated(DB::ExecutingGraph::Edge&, std::__1::queue > >&, std::__1::queue > >&, unsigned long) @ 0x1b0df2c1 in /usr/bin/clickhouse
2021.03.05 12:48:09.826054 [ 386235 ] {} BaseDaemon: 27. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:285: DB::PipelineExecutor::prepareProcessor(unsigned long, unsigned long, std::__1::queue > >&, std::__1::queue > >&, std::__1::unique_lock) @ 0x1b0df88b in /usr/bin/clickhouse
2021.03.05 12:48:10.409219 [ 386235 ] {} BaseDaemon: 28. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:188: DB::PipelineExecutor::tryAddProcessorToStackIfUpdated(DB::ExecutingGraph::Edge&, std::__1::queue > >&, std::__1::queue > >&, unsigned long) @ 0x1b0df2c1 in /usr/bin/clickhouse
2021.03.05 12:48:10.930219 [ 386235 ] {} BaseDaemon: 29. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:291: DB::PipelineExecutor::prepareProcessor(unsigned long, unsigned long, std::__1::queue > >&, std::__1::queue > >&, std::__1::unique_lock) @ 0x1b0df958 in /usr/bin/clickhouse
2021.03.05 12:48:11.527634 [ 386235 ] {} BaseDaemon: 30. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:188: DB::PipelineExecutor::tryAddProcessorToStackIfUpdated(DB::ExecutingGraph::Edge&, std::__1::queue > >&, std::__1::queue > >&, unsigned long) @ 0x1b0df2c1 in /usr/bin/clickhouse
2021.03.05 12:48:12.198590 [ 386235 ] {} BaseDaemon: 31. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:291: DB::PipelineExecutor::prepareProcessor(unsigned long, unsigned long, std::__1::queue > >&, std::__1::queue > >&, std::__1::unique_lock) @ 0x1b0df958 in /usr/bin/clickhouse
2021.03.05 12:48:12.825569 [ 386235 ] {} BaseDaemon: 32. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:188: DB::PipelineExecutor::tryAddProcessorToStackIfUpdated(DB::ExecutingGraph::Edge&, std::__1::queue > >&, std::__1::queue > >&, unsigned long) @ 0x1b0df2c1 in /usr/bin/clickhouse
2021.03.05 12:48:13.471791 [ 386235 ] {} BaseDaemon: 33. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:291: DB::PipelineExecutor::prepareProcessor(unsigned long, unsigned long, std::__1::queue > >&, std::__1::queue > >&, std::__1::unique_lock) @ 0x1b0df958 in /usr/bin/clickhouse
2021.03.05 12:48:14.184018 [ 386235 ] {} BaseDaemon: 34. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:614: DB::PipelineExecutor::executeStepImpl(unsigned long, unsigned long, std::__1::atomic*) @ 0x1b0e277e in /usr/bin/clickhouse
2021.03.05 12:48:14.802589 [ 386235 ] {} BaseDaemon: 35. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:473: DB::PipelineExecutor::executeSingleThread(unsigned long, unsigned long) @ 0x1b0e2f99 in /usr/bin/clickhouse
2021.03.05 12:48:15.413359 [ 386235 ] {} BaseDaemon: 36. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:812: DB::PipelineExecutor::executeImpl(unsigned long) @ 0x1b0e0ea4 in /usr/bin/clickhouse
2021.03.05 12:48:16.038803 [ 386235 ] {} BaseDaemon: 37. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:395: DB::PipelineExecutor::execute(unsigned long) @ 0x1b0e01a6 in /usr/bin/clickhouse
2021.03.05 12:48:16.472652 [ 386235 ] {} BaseDaemon: 38. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:80: DB::threadFunction(DB::PullingAsyncPipelineExecutor::Data&, std::__1::shared_ptr, unsigned long) @ 0x1b0fc3b7 in /usr/bin/clickhouse
2021.03.05 12:48:16.882691 [ 386235 ] {} BaseDaemon: 39. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PullingAsyncPipelineExecutor.cpp:108: DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0::operator()() const @ 0x1b0fc2f6 in /usr/bin/clickhouse
2021.03.05 12:48:17.290137 [ 386235 ] {} BaseDaemon: 40. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3682: decltype(std::__1::forward(fp)()) std::__1::__invoke_constexpr(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&) @ 0x1b0fc27d in /usr/bin/clickhouse
2021.03.05 12:48:19.347400 [ 278 ] {} Application: Child process was terminated by signal 6.
```

Related issue https://github.com/ClickHouse/ClickHouse/issues/21479

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.