Asan test failed
Open
Nobody has claimed this yet.
severity/minor
type/bug
- Dominant language
- C++
- Stars
- 1k
- Forks
- 423
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 24
Description
Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)
2. What did you expect to see? (Required)
3. What did you see instead (Required)
[2025-10-12T19:11:53.533Z] [2025/10/12 19:11:41.763 +00:00] [DEBUG] [Ctl.cpp:86] ["SYNC_FOR(in_FastAddPeerImplWrite::after_write_segments) trying... \n\n# Current Stack: \n 0x55844102d395\t___interceptor_backtrace [gtests_dbms+124142485]\n \t/tiflash-release-8.3/release-centos7-llvm/dockerfiles/misc/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:4355\n 0x55846582cae3\tStackTrace::StackTrace() [gtests_dbms+736508643]\n \tdbms/src/Common/StackTrace.cpp:23\n 0x5584659ce472\tDB::SyncPointCtl::sync(char const*) [gtests_dbms+738219122]\n \tdbms/src/Common/SyncPoint/Ctl.cpp:86\n 0x55845ad088e5\tDB::FastAddPeerImplWrite(DB::TMTContext&, DB::TiFlashRaftProxyHelper const*, unsigned long, unsigned long, std::__1::tuple<std::__1::shared_ptr<DB::CheckpointInfo>, std::__1::shared_ptr<DB::Region>, raft_serverpb::RaftApplyState, raft_serverpb::RegionLocalState>&&, unsigned long) [gtests_dbms+557054181]\n \tdbms/src/Storages/KVStore/MultiRaft/Disagg/FastAddPeer.cpp:460\n 0x55845ad0ca1e\tDB::FastAddPeerImpl(std::__1::shared_ptr<DB::FastAddPeerContext>, DB::TMTContext&, DB::TiFlashRaftProxyHelper const*, unsigned long, unsigned long, unsigned long) [gtests_dbms+557070878]\n \tdbms/src/Storages/KVStore/MultiRaft/Disagg/FastAddPeer.cpp:521\n 0x55845ad1a337\tstd::__1::__function::__func<FastAddPeer::$_5, std::__1::allocator<FastAddPeer::$_5>, DB::FastAddPeerRes ()>::operator()() [gtests_dbms+557126455]\n \tdbms/src/Storages/KVStore/MultiRaft/Disagg/FastAddPeer.cpp:781\n 0x55844a0ca99e\tstd::__1::__packaged_task_func<std::__1::function<DB::FastAddPeerRes ()>, std::__1::allocator<std::__1::function<DB::FastAddPeerRes ()>>, DB::FastAddPeerRes ()>::operator()() [gtests_dbms+275782046]\n \t/usr/local/bin/../include/c++/v1/__functional/function.h:517\n 0x55844a0cbbad\tstd::__1::packaged_task<DB::FastAddPeerRes ()>::operator()() [gtests_dbms+275786669]\n \t/usr/local/bin/../include/c++/v1/future:1891\n 0x55844a0caede\tDB::AsyncTasks<unsigned long, std::__1::function<DB::FastAddPeerRes ()>, DB::FastAddPeerRes>::addTaskWithCancel(unsigned long, std::__1::function<DB::FastAddPeerRes ()>, std::__1::function<void ()>)::'lambda'()::operator()() const [gtests_dbms+275783390]\n \tdbms/src/Storages/KVStore/Utils/AsyncTasks.h:300\n 0x558465946e95\tDB::ThreadPoolImpl<DB::ThreadFromGlobalPoolImpl<false>>::worker(std::__1::__list_iterator<DB::ThreadFromGlobalPoolImpl<false>, void*>) [gtests_dbms+737664661]\n \t/usr/local/bin/../include/c++/v1/__functional/function.h:517\n 0x558465952e9c\tstd::__1::__function::__func<DB::ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<bool DB::ThreadPoolImpl<DB::ThreadFromGlobalPoolImpl<false>>::scheduleImpl<bool>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()>(bool&&)::'lambda'(), std::__1::allocator<DB::ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<bool DB::ThreadPoolImpl<DB::ThreadFromGlobalPoolImpl<false>>::scheduleImpl<bool>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()>(bool&&)::'lambda'()>, void ()>::operator()() [gtests_dbms+737713820]\n \tdbms/src/Common/UniThreadPool.cpp:167\n 0x558465941ba2\tDB::ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) [gtests_dbms+737643426]\n \t/usr/local/bin/../include/c++/v1/__functional/function.h:517\n 0x55846594af03\tvoid* std::__1::__thread_proxy[abi:ue170006]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void DB::ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()>>(void*) [gtests_dbms+737681155]\n \tdbms/src/Common/UniThreadPool.cpp:167\n 0x55844108118b\tasan_thread_start(void*) [gtests_dbms+124486027]\n \t/tiflash-release-8.3/release-centos7-llvm/dockerfiles/misc/llvm-project/compiler-rt/lib/asan/asan_interceptors.cpp:225\n 0x7fd74770aea5\tstart_thread [libpthread.so.0+32421]\n 0x7fd747003b0d\tclone [libc.so.6+1043213]"] [source=SyncPointCtl] [thread_id=9]
[2025-10-12T19:11:53.534Z] [2025/10/12 19:11:41.763 +00:00] [DEBUG] [Ctl.cpp:92] ["SYNC_FOR(in_FastAddPeerImplWrite::after_write_segments) matched waitAndPause(), paused until calling next()..."] [source=SyncPointCtl] [thread_id=9]
[2025-10-12T19:11:53.534Z] [2025/10/12 19:11:41.763 +00:00] [DEBUG] [Ctl.cpp:71] ["waitAndPause(in_FastAddPeerImplWrite::after_write_segments) finished"] [source=SyncPointCtl] [thread_id=1]
[2025-10-12T19:11:53.534Z] [2025/10/12 19:11:41.763 +00:00] [DEBUG] [Ctl.cpp:77] ["next(in_FastAddPeerImplWrite::after_write_segments) trying..."] [source=SyncPointCtl] [thread_id=1]
[2025-10-12T19:11:53.534Z] [2025/10/12 19:11:41.763 +00:00] [DEBUG] [Ctl.cpp:96] ["SYNC_FOR(in_FastAddPeerImplWrite::after_write_segments) done"] [source=SyncPointCtl] [thread_id=9]
[2025-10-12T19:11:53.534Z] [2025/10/12 19:11:41.763 +00:00] [DEBUG] [Ctl.cpp:79] ["next(in_FastAddPeerImplWrite::after_write_segments) done"] [source=SyncPointCtl] [thread_id=1]
[2025-10-12T19:11:53.534Z] [2025/10/12 19:11:41.763 +00:00] [DEBUG] [FastAddPeer.cpp:480] ["Finish write FAP snapshot, log_count=0"] [source="FastAddPeer(region_id=1 keyspace=4294967295 table_id=31 new_peer_id=2333)"] [thread_id=9]
[2025-10-12T19:11:53.534Z] [2025/10/12 19:11:41.763 +00:00] [DEBUG] [Ctl.cpp:50] ["Disabled: in_FastAddPeerImplWrite::after_write_segments"] [source=SyncPointCtl] [thread_id=1]
[2025-10-12T19:11:53.534Z] [2025/10/12 19:11:41.763 +00:00] [INFO] [FastAddPeer.cpp:829] ["Fetch task, result=Ok new_peer_id=2333 region_id=1 elapsed=18"] [source=FastAddPeer] [thread_id=1]
[2025-10-12T19:11:53.534Z] [2025/10/12 19:11:51.765 +00:00] [DEBUG] [FastAddPeerCache.h:76] ["ParsedCheckpointDataHolder destroyed path=/tiflash/tmp/55638/page/local_100"] [thread_id=1]
[2025-10-12T19:11:53.534Z] /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/KVStore/tests/gtest_kvstore_fast_add_peer.cpp:952: Failure
[2025-10-12T19:11:53.534Z] Failed
[2025-10-12T19:11:53.534Z] Code: 0. DB::Exception: not meet
[2025-10-12T19:11:53.534Z]
[2025-10-12T19:11:53.534Z] Stack trace:
[2025-10-12T19:11:53.534Z]
[2025-10-12T19:11:53.534Z] 0x55844102d395 ___interceptor_backtrace [gtests_dbms+124142485]
[2025-10-12T19:11:53.534Z] /tiflash-release-8.3/release-centos7-llvm/dockerfiles/misc/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:4355
[2025-10-12T19:11:53.534Z] 0x55846582cae3 StackTrace::StackTrace() [gtests_dbms+736508643]
[2025-10-12T19:11:53.534Z] dbms/src/Common/StackTrace.cpp:23
[2025-10-12T19:11:53.534Z] 0x55844a09f88c DB::tests::RegionKVStoreTestFAP_Cancel4_Test::TestBody() [gtests_dbms+275605644]
[2025-10-12T19:11:53.534Z] dbms/src/Common/Exception.h:46
[2025-10-12T19:11:53.534Z] 0x55845af954f2 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) [gtests_dbms+559727858]
[2025-10-12T19:11:53.534Z] contrib/googletest/googletest/src/gtest.cc:2443
[2025-10-12T19:11:53.534Z] 0x55845af35b36 testing::Test::Run() [gtests_dbms+559336246]
[2025-10-12T19:11:53.534Z] contrib/googletest/googletest/src/gtest.cc:2517
[2025-10-12T19:11:53.534Z] 0x55845af3909b testing::TestInfo::Run() [gtests_dbms+559349915]
[2025-10-12T19:11:53.534Z] contrib/googletest/googletest/src/gtest.cc:2693
[2025-10-12T19:11:53.534Z] 0x55845af3a750 testing::TestCase::Run() [gtests_dbms+559355728]
[2025-10-12T19:11:53.534Z] contrib/googletest/googletest/src/gtest.cc:2811
[2025-10-12T19:11:53.534Z] 0x55845af65c4b testing::internal::UnitTestImpl::RunAllTests() [gtests_dbms+559533131]
[2025-10-12T19:11:53.534Z] contrib/googletest/googletest/src/gtest.cc:5177
[2025-10-12T19:11:53.534Z] 0x55845af97c32 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) [gtests_dbms+559737906]
[2025-10-12T19:11:53.534Z] contrib/googletest/googletest/src/gtest.cc:2443
[2025-10-12T19:11:53.534Z] 0x55845af64ee7 testing::UnitTest::Run() [gtests_dbms+559529703]
[2025-10-12T19:11:53.534Z] contrib/googletest/googletest/src/gtest.cc:4786
[2025-10-12T19:11:53.534Z] 0x55844b1dd936 main [gtests_dbms+293685558]
[2025-10-12T19:11:53.534Z] contrib/googletest/googletest/include/gtest/gtest.h:2341
[2025-10-12T19:11:53.534Z] 0x7fd746f27555 __libc_start_main [libc.so.6+140629]
[2025-10-12T19:11:53.534Z] 0x558440fe7629 <unknown symbol> [gtests_dbms+123856425]
[2025-10-12T19:11:53.534Z]
[2025-10-12T19:11:53.534Z] [2025/10/12 19:11:51.766 +00:00] [INFO] [AsyncTasks.h:105] ["Pending 0 tasks when destructing"] [thread_id=1]
[2025-10-12T19:11:53.534Z] [2025/10/12 19:11:51.766 +00:00] [INFO] [AsyncTasks.h:109] ["Finish finalize thread pool"] [thread_id=1]
[2025-10-12T19:11:53.534Z] [2025/10/12 19:11:51.766 +00:00] [WARN] [Context.cpp:1799] ["Release GlobalUniversalPageStorage(WriteNode)."] [source=Context] [thread_id=1]
[2025-10-12T19:11:53.534Z] [2025/10/12 19:11:51.767 +00:00] [DEBUG] [TiFlashTestEnv.cpp:288] ["Created bucket mockbucket"] [source="bucket=mockbucket root=tiflash_ut/"] [thread_id=1]
[2025-10-12T19:11:53.534Z] [2025/10/12 19:11:51.767 +00:00] [WARN] [TiFlashTestEnv.cpp:336] ["DeleteBucket: :"] [source="bucket=mockbucket root=tiflash_ut/"] [thread_id=1]
[2025-10-12T19:11:53.534Z] [0;31m[ FAILED ] [mRegionKVStoreTestFAP.Cancel4 (11096 ms)
[2025-10-12T19:11:53.534Z] [0;32m[----------] [m1 test from RegionKVStoreTestFAP (11096 ms total)
[2025-10-12T19:11:53.534Z]
[2025-10-12T19:11:53.534Z] [0;32m[----------] [mGlobal test environment tear-down
[2025-10-12T19:11:53.534Z] [0;32m[==========] [m1 test from 1 test case ran. (11096 ms total)
[2025-10-12T19:11:53.534Z] [0;32m[ PASSED ] [m0 tests.
[2025-10-12T19:11:53.534Z] [0;31m[ FAILED ] [m1 test, listed below:
[2025-10-12T19:11:53.534Z] [0;31m[ FAILED ] [mRegionKVStoreTestFAP.Cancel4
[2025-10-12T19:11:53.534Z]
[2025-10-12T19:11:53.534Z] 1 FAILED TEST
4. What is your TiFlash version? (Required)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with dbms/src/Storages/KVStore/tests/gtest_kvstore_fast_add_peer.cpp at line 952 and the RegionKVStoreTestFAP.Cancel4 test. Review the FastAddPeer.cpp synchronization and cancellation paths referenced in the stack trace, then reproduce the ASAN failure. Done means identifying and fixing the cause of the DB::Exception: not meet failure and making Cancel4 pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100