[VL] Executor crash frequently occur in version 1.3.0

### Backend

VL (Velox)

### Bug description

1.3.0

2024-10-09

I used the main version of 2024-10-09 and 1.3.0 to run the same series of tasks to test the stability and performance of version 1.3.0. During the test, I found that the executor of 1.3.0 frequently crashed.
executor log
```
pure virtual method called
terminate called without an active exception
```
core dump
```
(gdb) bt
#0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=139908111537728) at ./nptl/pthread_kill.c:44
#1 __pthread_kill_internal (signo=6, threadid=139908111537728) at ./nptl/pthread_kill.c:78
#2 __GI___pthread_kill (threadid=139908111537728, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3 0x00007f3f99233476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4 0x00007f3f992197f3 in __GI_abort () at ./stdlib/abort.c:79
#5 0x00007f3f98f80b9e in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#6 0x00007f3f98f8c20c in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#7 0x00007f3f98f8c277 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#8 0x00007f3f98f8cfa5 in __cxa_pure_virtual () from /lib/x86_64-linux-gnu/libstdc++.so.6
#9 0x00007f3ee6c5634b in gluten::ListenableArbitrator::growCapacityInternal(facebook::velox::memory::MemoryPool*, unsigned long) ()
from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
#10 0x00007f3ee6c56993 in gluten::ListenableArbitrator::growCapacity(facebook::velox::memory::MemoryPool*, unsigned long) ()
from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
#11 0x00007f3ee6e29797 in facebook::velox::memory::MemoryPoolImpl::growCapacity(facebook::velox::memory::MemoryPool*, unsigned long) ()
from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
#12 0x00007f3ee6e299f3 in facebook::velox::memory::MemoryPoolImpl::incrementReservationThreadSafe(facebook::velox::memory::MemoryPool*, unsigned long) ()
from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
#13 0x00007f3ee6e299b9 in facebook::velox::memory::MemoryPoolImpl::incrementReservationThreadSafe(facebook::velox::memory::MemoryPool*, unsigned long) ()
from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
--Type for more, q to quit, c to continue without paging--c
#14 0x00007f3ee6e299b9 in facebook::velox::memory::MemoryPoolImpl::incrementReservationThreadSafe(facebook::velox::memory::MemoryPool*, unsigned long) () from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
#15 0x00007f3ee6e299b9 in facebook::velox::memory::MemoryPoolImpl::incrementReservationThreadSafe(facebook::velox::memory::MemoryPool*, unsigned long) () from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
#16 0x00007f3ee6e2b66c in facebook::velox::memory::MemoryPoolImpl::reserveThreadSafe(unsigned long, bool) () from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
#17 0x00007f3ee6e1c19d in facebook::velox::memory::MemoryAllocator::allocateNonContiguous(unsigned long, facebook::velox::memory::Allocation&, std::function, unsigned long) () from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
#18 0x00007f3ee6e302a9 in facebook::velox::memory::MemoryPoolImpl::allocateNonContiguous(unsigned long, facebook::velox::memory::Allocation&, unsigned long) () from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
#19 0x00007f3eea303d5b in facebook::velox::dwio::common::DirectCoalescedLoad::loadData(bool) () from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
#20 0x00007f3ee6ddb026 in facebook::velox::cache::CoalescedLoad::loadOrFuture(folly::SemiFuture*, bool) () from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
#21 0x00007f3eea302269 in void folly::detail::function::call_ >, bool)::{lambda()#8}, true, false, void>(, folly::detail::function::Data&) () from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
#22 0x00007f3eea6b71f4 in folly::detail::function::FunctionTraits::operator()() (this=0x7f3ee549f010) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/Function.h:370
#23 folly::catch_exception&, void (&)(char const*) noexcept, char const*&, void>(folly::Function&, void (&)(char const*) noexcept, char const*&) (c=, t=...) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/lang/Exception.h:290
#24 folly::Executor::invokeCatchingExns >(char const*, folly::Function) (f=..., p=0x7f3eeacbfa43 "ThreadPoolExecutor: func") at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/Executor.h:234
#25 folly::ThreadPoolExecutor::runTask (this=0x7f3f004e3380, thread=std::shared_ptr (use count 8, weak count 0) = {...}, task=...) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/executors/ThreadPoolExecutor.cpp:142
#26 0x00007f3eea6ad07b in operator() (__closure=0x7f3f025400a0) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/executors/IOThreadPoolExecutor.cpp:149
#27 folly::detail::function::call_, false, false, void>(folly::detail::function::Data &) (p=...) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/Function.h:341
#28 0x00007f3eea6f5e66 in folly::detail::function::FunctionTraits::operator()() (this=0x7f3ee549f160) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/Function.h:368
#29 folly::EventBase::FuncRunner::operator()(folly::Function&&) (func=..., this=0x7f3f05e055c0) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/io/async/EventBase.cpp:200
#30 folly::detail::invokeConsumerWithTask, folly::EventBase::FuncRunner&, void>(folly::EventBase::FuncRunner&, folly::Function&&, std::shared_ptr&&) (rctx=..., task=..., consumer=...) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/io/async/AtomicNotificationQueue-inl.h:254
#31 folly::AtomicNotificationQueue >::drive(folly::EventBase::FuncRunner&) (this=0x7f3f05e05500, consumer=...) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/io/async/AtomicNotificationQueue-inl.h:337
#32 0x00007f3eea6f764d in non-virtual thunk to folly::EventBaseAtomicNotificationQueue, folly::EventBase::FuncRunner>::handlerReady(unsigned short) () from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
#33 0x00007f3eea6fe1e8 in folly::EventHandler::libeventCallback (fd=, events=, arg=0x7f3f05e05428) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/io/async/EventHandler.cpp:159
#34 0x00007f3eea851fb6 in event_process_active_single_queue () from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
#35 0x00007f3eea8529ef in event_base_loop () from /tmp/spark-1decd196-73b9-44d2-ac0a-6d376303fac7/executor-f24aeb44-8369-47b5-b6d4-971202f55a9e/gluten-9f3ff136-1ab4-4207-8126-f937a4438b27/jni/9d971c63-f9a3-4dd2-a99b-70f50853eccd/gluten-14997099487791574374/linux/amd64/libvelox.so
#36 0x00007f3eea6f077e in (anonymous namespace)::EventBaseBackend::eb_event_base_loop (flags=1, this=) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/io/async/EventBase.cpp:99
#37 folly::EventBase::loopMain (this=0x7f3f05e69280, flags=, options=...) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/io/async/EventBase.cpp:603
#38 0x00007f3eea6f0e01 in folly::EventBase::loopBody (this=0x7f3f05e69280, flags=0, options=...) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/io/async/EventBase.cpp:513
#39 0x00007f3eea6f0e86 in folly::EventBase::loop (this=0x7f3f05e69280) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/io/async/EventBase.cpp:474
#40 0x00007f3eea6f2f99 in folly::EventBase::loopForever (this=0x7f3f05e69280) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/io/async/EventBase.cpp:781
#41 0x00007f3eea6ae08c in folly::IOThreadPoolExecutor::threadRun (this=0x7f3f004e3380, thread=...) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/executors/IOThreadPoolExecutor.cpp:240
#42 0x00007f3eea6bdf19 in std::__invoke_impl), folly::ThreadPoolExecutor*&, std::shared_ptr&> (__f=, __t=, __f=, __t=) at /opt/rh/devtoolset-11/root/usr/include/c++/11/bits/invoke.h:74
#43 std::__invoke), folly::ThreadPoolExecutor*&, std::shared_ptr&> (__fn=) at /opt/rh/devtoolset-11/root/usr/include/c++/11/bits/invoke.h:96
#44 std::_Bind))(std::shared_ptr)>::__call(std::tuple<>&&, std::_Index_tuple<0ul, 1ul>) (__args=..., this=) at /opt/rh/devtoolset-11/root/usr/include/c++/11/functional:420
#45 std::_Bind))(std::shared_ptr)>::operator()<, void>() (this=) at /opt/rh/devtoolset-11/root/usr/include/c++/11/functional:503
#46 folly::detail::function::call_))(std::shared_ptr)>, true, false, void>(, folly::detail::function::Data&) (p=...) at /opt/gluten/dev/vcpkg/.vcpkg/buildtrees/folly/src/4.07.01.00-4ae9f6791f.clean/folly/Function.h:341
#47 0x00007f3f98fba253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#48 0x00007f3f99285ac3 in start_thread (arg=) at ./nptl/pthread_create.c:442
#49 0x00007f3f99316a04 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100
(gdb)
```
Monitoring by spark metrics shows that the memory is sufficient.
conf

metrics

### Spark version

Spark-3.4.x

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

```bash

```