apache / apache/doris

[Bug] BE crashed with SIGSEGV and auto-restarted, causing query disruption while it recovered.

Open
#65,773 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.9k
Forks
3.9k
Avg merge
2d 23h
Merged PRs (30d)
520

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.

### Version

4.0.7-rc02-35854e7e92a

### What's Wrong?

grep -A 15 "1784283889" be.out
*** Aborted at 1784283889 (unix time) try "date -d @1784283889" if you are using GNU date ***
*** Current BE git commitID: 35854e7e92a ***
*** SIGSEGV address not mapped to object (@0x1f8) received by PID 2929477 (TID 3000255 OR 0x7d4622d7b6c0) from PID 504; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_release/doris/be/src/common/signal_handler.h:420
1# PosixSignals::chained_handler(int, siginfo_t*, void*) in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
3# 0x00007D6150845330 in /lib/x86_64-linux-gnu/libc.so.6
4# doris::RuntimeState::is_nereids() const at /home/zcp/repo_center/doris_release/doris/be/src/runtime/runtime_state.cpp:511
5# doris::AttachTask::AttachTask(doris::RuntimeState*) at /home/zcp/repo_center/doris_release/doris/be/src/runtime/thread_context.cpp:47
6# std::_Function_handler::_M_invoke(std::_Any_data const&) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292
7# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:623
8# doris::Thread::supervise_thread(void*) at /home/zcp/repo_center/doris_release/doris/be/src/util/thread.cpp:461
9# 0x00007D615089CAA4 in /lib/x86_64-linux-gnu/libc.so.6
10# 0x00007D6150929C6C in /lib/x86_64-linux-gnu/libc.so.6

INFO: java_cmd /usr/lib/jvm/java-17-openjdk-amd64/bin/java

### What You Expected?

No BE crash

### How to Reproduce?

Not yet isolated into a minimal repro. Observed in production: BE crashed with the above
trace while running queries against a JDBC External Catalog (MS SQL Server). The crash
occurs inside the async result-writer's thread pool dispatch, when constructing an
AttachTask from a RuntimeState pointer — consistent with the RuntimeState having already
been destroyed by that point (use-after-free)

### Anything Else?

_No response_

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Research direction

Start with be/src/runtime/runtime_state.cpp:511 and be/src/runtime/thread_context.cpp:47, then trace vectorized::AsyncResultWriter::start_writer and the ThreadPool dispatch path shown in the stack trace. Investigate whether the RuntimeState passed to AttachTask can be destroyed before the async writer runs. Done means reproducing or validating the lifetime failure and preventing the BE SIGSEGV without disrupting query recovery.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, java, sql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.