Core dump in __gnu_cxx::__ops::_Val_comp_iter<oceanbase::sql::ObSortOpImpl::CopyableComparer>::operator() during async index operation

Environment

• Version: observer (OceanBase seekdb)
• Source Revision: 1-f8501118d6eca3ee5c2714151d0a5cf842726dcb
• Build Branch: master
• Build Time: Apr 9 2026 23:40:49
• Build Flags: RelWithDebInfo|Sanity
• Build Info: obbuild-sanity-master-100905

Description

A core dump occurred during an operation related to asynchronous index creation. The crash originates from a memory sanity check abort within the std::sort operation, specifically in the comparator __gnu_cxx::__ops::_Val_comp_iter<oceanbase::sql::ObSortOpImpl::CopyableComparer>::operator().

Steps to Reproduce

The specific workload and actions leading to the crash were not detailed in the original report.

Impact

The business impact was not specified.

Log Information

Core Stack Trace

#0  0x00007f3ea76c4055 in raise  from /lib64/libpthread.so.0
#1  0x000055dbb345d9a8 in oceanbase::common::coredump_cb (sig=6, sig_code=-6, sig_addr=0x3f5002e27d5, context=<optimized out>) at ./deps/oblib/src/lib/signal/ob_signal_handlers.cpp:256
#2  <signal handler called>
#3  0x00007f3ea7526005 in raise  from /lib64/libc.so.6
#4  0x00007f3ea74f8894 in abort  from /lib64/libc.so.6
#5  0x000055db96b05079 in memory_sanity_abort  at ./deps/oblib/src/lib/alloc/memory_sanity.cpp:547
#6  0x000055dba01056f0 in __gnu_cxx::__ops::_Val_comp_iter<oceanbase::sql::ObSortOpImpl::CopyableComparer>::operator<oceanbase::sql::ObChunkDatumStore::StoredRow*, oceanbase::lib::IteratorWrapper<oceanbase::sql::ObChunkDatumStore::StoredRow**> > (this=0x7f3e929cd9d8, __val=<optimized out>, __it=...) at ./deps/3rd/usr/local/oceanbase/devtools/lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/bits/predefined_ops.h:240
#7  0x000055dba0105425 in std::__unguarded_linear_insert<oceanbase::lib::IteratorWrapper<oceanbase::sql::ObChunkDatumStore::StoredRow**>, __gnu_cxx::__ops::_Val_comp_iter<oceanbase::sql::ObSortOpImpl::CopyableComparer> > (__last=..., __comp=...) at ./deps/3rd/usr/local/oceanbase/devtools/lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/bits/stl_algo.h:1789
#8  0x000055dba0103296 in std::__unguarded_insertion_sort<oceanbase::lib::IteratorWrapper<oceanbase::sql::ObChunkDatumStore::StoredRow**>, __gnu_cxx::__ops::_Iter_comp_iter<oceanbase::sql::ObSortOpImpl::CopyableComparer> > (__last=..., __comp=..., __first=...) at ./deps/3rd/usr/local/oceanbase/devtools/lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/bits/stl_algo.h:1830
#9  std::__final_insertion_sort<oceanbase::lib::IteratorWrapper<oceanbase::sql::ObChunkDatumStore::StoredRow**>, __gnu_cxx::__ops::_Iter_comp_iter<oceanbase::sql::ObSortOpImpl::CopyableComparer> > (__first=..., __last=..., __comp=...) at ./deps/3rd/usr/local/oceanbase/devtools/lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/bits/stl_algo.h:1850
#10 0x000055dba0102bc2 in std::__sort<oceanbase::lib::IteratorWrapper<oceanbase::sql::ObChunkDatumStore::StoredRow**>, __gnu_cxx::__ops::_Iter_comp_iter<oceanbase::sql::ObSortOpImpl::CopyableComparer> > (__first=..., __last=..., __comp=...) at ./deps/3rd/usr/local/oceanbase/devtools/lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/bits/stl_algo.h:1940
#11 0x000055db9ff5b290 in std::sort<oceanbase::lib::IteratorWrapper<oceanbase::sql::ObChunkDatumStore::StoredRow**>, oceanbase::sql::ObSortOpImpl::CopyableComparer> (__first=..., __last=..., __comp=...) at ./deps/3rd/usr/local/oceanbase/devtools/lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/bits/stl_algo.h:4853
#12 oceanbase::lib::ob_sort<oceanbase::sql::ObChunkDatumStore::StoredRow**, oceanbase::sql::ObSortOpImpl::CopyableComparer> (first=<optimized out>, last=<optimized out>, comp=...) at ./deps/oblib/src/lib/utility/ob_sort.h:177
#13 0x000055db9ff5dec8 in oceanbase::sql::ObSortOpImpl::sort_inmem_data (this=0xa0085bfc340) at ./src/sql/engine/sort/ob_sort_op_impl.cpp:1984
#14 0x000055db9ff21df9 in oceanbase::sql::ObSortOpImpl::sort (this=0xa0085bfc340) at ./src/sql/engine/sort/ob_sort_op_impl.cpp:2053
#15 0x000055db9ff26ac1 in oceanbase::sql::ObSortOp::process_sort_batch (this=0xa0085bfc060) at ./src/sql/engine/sort/ob_sort_op.cpp:293
#16 0x000055db9ff30b6f in oceanbase::sql::ObSortOp::inner_get_next_batch (this=<optimized out>, max_row_cnt=256) at ./src/sql/engine/sort/ob_sort_op.cpp:571
#17 0x000055db9fe44a39 in oceanbase::sql::ObOperator::get_next_batch (this=0xa0085bfc060, max_row_cnt=<optimized out>, batch_rows=<optimized out>) at ./src/sql/engine/ob_operator.cpp:1480
#18 0x000055db9fe5a4f3 in oceanbase::sql::ObBatchRowIter::get_next_row (this=0xa0035ae77e0, eval_ctx=..., spec=...) at ./src/sql/engine/ob_operator.cpp:2032
#19 0x000055db9fe42a61 in oceanbase::sql::ObOperator::get_next_row_vectorizely (this=0xa0085bfc060) at ./src/sql/engine/ob_operator.cpp:1912
#20 0x000055db9fe40b53 in oceanbase::sql::ObOperator::get_next_row (this=0xa0085bfc060) at ./src/sql/engine/ob_operator.cpp:1288
#21 0x000055dba01e00b6 in oceanbase::sql::ObPxMultiPartSSTableInsertOp::get_next_row_from_child (this=0xa004cf5a060, insert_monitor=0x7f3e929cebe8) at ./src/sql/engine/pdml/static/ob_px_sstable_insert_op.cpp:255
#22 0x000055dba01de805 in oceanbase::sql::ObPxMultiPartSSTableInsertOp::write_ordered_slice_by_row (this=0xa004cf5a060) at ./src/sql/engine/pdml/static/ob_px_sstable_insert_op.cpp:841
#23 0x000055dba01dd129 in oceanbase::sql::ObPxMultiPartSSTableInsertOp::inner_get_next_row (this=0xa004cf5a060) at ./src/sql/engine/pdml/static/ob_px_sstable_insert_op.cpp:211
#24 0x000055db9fe3ee1b in oceanbase::sql::ObOperator::get_next_row (this=0xa004cf5a070) at ./src/sql/engine/ob_operator.cpp:1306
#25 0x000055db9fe4a012 in oceanbase::sql::ObOperator::get_next_batch_with_onlyone_row (this=0xa004cf5a070) at ./src/sql/engine/ob_operator.h:541
#26 0x000055db9fe48690 in oceanbase::sql::ObOperator::get_next_batch (this=0xa004cf5a070, max_row_cnt=256, batch_rows=@0x7f3e929cf488: 0xa004cf5a2d0) at ./src/sql/engine/ob_operator.cpp:1595
#27 0x000055db9ba7a051 in oceanbase::sql::ObPxTransmitOp::next_vector (this=0xa00db5fc060, max_row_cnt=<optimized out>) at ./src/sql/engine/px/exchange/ob_px_transmit_op.cpp:482
#28 0x000055db9fe44a39 in oceanbase::sql::ObOperator::get_next_batch (this=0xa00db5fc060, max_row_cnt=<optimized out>, batch_rows=<optimized out>) at ./src/sql/engine/ob_operator.cpp:1480
#29 0x000055db9ba749c6 in oceanbase::sql::ObPxTransmitOp::fetch_first_row (this=<optimized out>) at ./src/sql/engine/px/exchange/ob_px_transmit_op.cpp:267
#30 0x000055db9b98a665 in oceanbase::sql::ObPxTransmitOp::inner_open (this=0xa00db5fc060) at ./src/sql/engine/px/exchange/ob_px_transmit_op.cpp:215
#31 0x000055db9ba36b69 in oceanbase::sql::ObPxReduceTransmitOp::inner_open (this=0xa00db5fc060) at ./src/sql/engine/px/exchange/ob_px_reduce_transmit_op.cpp:38
#32 0x000055db9fe35d0a in oceanbase::sql::ObOperator::open (this=0xa00db5fc060) at ./src/sql/engine/ob_operator.cpp:889
#33 0x000055dba0e228fc in oceanbase::sql::ObPxTaskProcess::execute (this=<optimized out>, root_spec=...) at ./src/sql/engine/px/ob_px_task_process.cpp:314
#34 0x000055dba0e1f32e in oceanbase::sql::ObPxTaskProcess::do_process (this=0x7f3e929d13a0) at ./src/sql/engine/px/ob_px_task_process.cpp:485
#35 0x000055dba0e1b1a1 in oceanbase::sql::ObPxTaskProcess::process (this=0x7f3e929d13a0) at ./src/sql/engine/px/ob_px_task_process.cpp:176
#36 0x000055dba0e18ea4 in oceanbase::sql::ObPxTaskProcess::run (this=0x7f3e929d13a0) at ./src/sql/engine/px/ob_px_task_process.cpp:104
#37 0x000055dba0e6ee7f in oceanbase::sql::PxWorkerFunctor::operator (this=0xa0075cca040, need_exec=<optimized out>) at ./src/sql/engine/px/ob_px_worker.cpp:220
#38 0x000055db99b97a50 in std::function<void (bool)>::operator(bool) const (this=<optimized out>, __args=true) at ./deps/3rd/usr/local/oceanbase/devtools/lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/bits/std_function.h:591
#39 oceanbase::omt::ObPxPool::handle (this=0xa0014ca0030, task=0xa0000eaa030) at ./src/observer/omt/ob_tenant.cpp:236
#40 0x000055db99b986df in oceanbase::omt::ObPxPool::run1 (this=0xa0014ca0030) at ./src/observer/omt/ob_tenant.cpp:289
#41 0x000055db99b97fd4 in oceanbase::omt::ObPxPool::run (this=0xa0014ca0030, idx=13) at ./src/observer/omt/ob_tenant.cpp:257
#42 0x000055dbb2389cbd in oceanbase::lib::Thread::run (this=<optimized out>) at ./deps/oblib/src/lib/thread/thread.cpp:228
#43 0x000055dbb238817a in oceanbase::lib::Thread::__th_start (arg=<optimized out>) at ./deps/oblib/src/lib/thread/thread.cpp:448
#44 0x00007f3ea76b93fb in start_thread  from /lib64/libpthread.so.0
#45 0x00007f3ea750fe83 in clone  from /lib64/libc.so.6

Additional Context

• Is this an RD-provided temporary version?: No/xxx temporary provision.
• Reproducibility: Deterministic/Intermittent/Uncertain/Other.
• Test Changes: Added xx test scenario/Changed to a new machine/Ran xx new tool/Issue appeared after replacing with a new version.
• Other anomalies or changes in the environment: No/Clog disk was full/Manual operation performed/xx exception was injected.
• Environment Information: http://:4105/obdata/data/ddltmp/obm.z1.obs0/
• Core Location: /obdata/data

Start with src/sql/engine/sort/ob_sort_op_impl.cpp at sort_inmem_data and deps/oblib/src/lib/utility/ob_sort.h, then trace the comparator and memory-sanity abort in the provided stack. Reconstruct an asynchronous index-creation workload and determine the failure cause; done means the workload no longer produces the core dump and the behavior is covered by a regression test.