apache / apache/incubator-pegasus
Assert from rocksdb posix_logger
- Dominant language
- C++
- Stars
- 2.1k
- Forks
- 328
- PR merge metrics
- No merged PRs in 30d
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. What did you do?
If possible, provide a recipe for reproducing the error.
Normal rolling-update on an existing cluster. (c4tst-perfomance)
### 2. What did you expect to see?
No error.
### 3. What did you see instead?
pegasus_server: /home/wutao1/pegasus/rocksdb/env/posix_logger.h:151: virtual void rocksdb::PosixLogger::Logv(const char*, __va_list_tag*): Assertion `sz == write_size' failed.
```
(gdb) bt
#0 0x00007f7845b641d7 in raise () from /lib64/libc.so.6
#1 0x00007f7845b658c8 in abort () from /lib64/libc.so.6
#2 0x00007f7845b5d146 in __assert_fail_base () from /lib64/libc.so.6
#3 0x00007f7845b5d1f2 in __assert_fail () from /lib64/libc.so.6
#4 0x0000000000924572 in rocksdb::PosixLogger::Logv(char const*, __va_list_tag*) ()
#5 0x0000000000896cb3 in rocksdb::Log(rocksdb::InfoLogLevel, rocksdb::Logger*, char const*, ...) ()
#6 0x00000000008a35ac in rocksdb::ImmutableDBOptions::Dump(rocksdb::Logger*) const ()
#7 0x00000000007e6fd3 in rocksdb::DBImpl::DBImpl(rocksdb::DBOptions const&, std::string const&) ()
#8 0x000000000081c0ef in rocksdb::DB::Open(rocksdb::DBOptions const&, std::string const&, std::vector > const&, std::vector >*, rocksdb::DB**) ()
#9 0x000000000081d856 in rocksdb::DB::Open(rocksdb::Options const&, std::string const&, rocksdb::DB**) ()
#10 0x0000000000701e98 in pegasus::server::pegasus_server_impl::start (this=0x57d0000, argc=, argv=)
at /home/wutao1/pegasus/src/server/pegasus_server_impl.cpp:1515
#11 0x00007f784964e08b in dsn::replication::replication_app_base::open (this=this@entry=0x57d0000)
at /home/wutao1/pegasus/rdsn/src/dist/replication/lib/replication_app_base.cpp:425
#12 0x00007f7849650968 in dsn::replication::replication_app_base::open_internal (this=0x57d0000, r=r@entry=0x2f6f700)
at /home/wutao1/pegasus/rdsn/src/dist/replication/lib/replication_app_base.cpp:346
#13 0x00007f784960e0af in dsn::replication::replica::init_app_and_prepare_list (this=this@entry=0x2f6f700, create_new=create_new@entry=false)
at /home/wutao1/pegasus/rdsn/src/dist/replication/lib/replica_init.cpp:211
#14 0x00007f784960f45a in dsn::replication::replica::initialize_on_load (this=this@entry=0x2f6f700) at /home/wutao1/pegasus/rdsn/src/dist/replication/lib/replica_init.cpp:121
#15 0x00007f784960f88a in dsn::replication::replica::load (stub=0x27de580, dir=0x2e185b8 "/home/work/ssd6/pegasus/c4tst-perfomance/replica/reps/60.33.pegasus")
at /home/wutao1/pegasus/rdsn/src/dist/replication/lib/replica_init.cpp:162
#16 0x00007f784962e801 in operator() (__closure=0x2cf8fe0) at /home/wutao1/pegasus/rdsn/src/dist/replication/lib/replica_stub.cpp:397
#17 std::_Function_handler::_M_invoke(const std::_Any_data &) (
__functor=...) at /home/wutao1/app/include/c++/4.8.2/functional:2071
#18 0x00007f784970b149 in dsn::task::exec_internal (this=this@entry=0x2e1ec60) at /home/wutao1/pegasus/rdsn/src/core/core/task.cpp:180
#19 0x00007f784971eedd in dsn::task_worker::loop (this=0x2c704d0) at /home/wutao1/pegasus/rdsn/src/core/core/task_worker.cpp:211
#20 0x00007f784971f0a9 in dsn::task_worker::run_internal (this=0x2c704d0) at /home/wutao1/pegasus/rdsn/src/core/core/task_worker.cpp:191
#21 0x00007f78464bc600 in std::(anonymous namespace)::execute_native_thread_routine (__p=)
at /home/qinzuoyan/git.xiaomi/pegasus/toolchain/objdir/../gcc-4.8.2/libstdc++-v3/src/c++11/thread.cc:84
#22 0x00007f784712fdc5 in start_thread () from /lib64/libpthread.so.0
#23 0x00007f7845c2673d in clone () from /lib64/libc.so.6
```
### 4. What version of Pegasus are you using?
pegasus (bdc029937bee99025c69b6a7f71ef4d31a1c5abb)
rocksdb (dc91f8a9567b65cea9cc7d1cc792ddd764141638)
The cause for this error may be the incompatible upgrade of rocksdb.
Contributor guide
Research direction
Start at rocksdb/env/posix_logger.h:151 and follow the stack through ImmutableDBOptions::Dump and pegasus_server_impl.cpp:1515. Compare the listed Pegasus and RocksDB revisions, reproduce the assertion during a rolling update if possible, and establish a verified fix or compatibility requirement.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100