apache / apache/doris

[Bug] Restarting the doris-disaggregated be pod failed.

Open
#58,083 7 comments 0 reactions 0 assignees View on GitHub
k8s
Dominant language
Java
Stars
15.9k
Forks
3.9k
Avg merge
2d 23h
Merged PRs (30d)
520

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.

### Version

image: apache/doris:be-3.1.1

### What's Wrong?

## I deployed doris-disaggregated in Kubernetes, but restarting the be pod failed.

```
20251117 06:47:10.896448 171 exec_env_init.cpp:201] Using SimdJSON Haswell implementation
I20251117 06:47:11.852528 171 wal_manager.cpp:489] Scheduled(every 10s) WAL info: [/opt/apache-doris/be/file_cache/wal: limit 4254319411 Bytes, used 0 Bytes, estimated wal bytes 0 Bytes, available 4254319411 Bytes.];
I20251117 06:47:12.896437 171 spill_stream_manager.cpp:211] spill gc time: 174.854us, spill storage:
path: /opt/apache-doris/be/file_cache, capacity: 39.94 GB, limit: 7.19 GB, used: 0, available: 39.62 GB
I20251117 06:47:13.352880 171 agent_server.cpp:60] Register workload group listener
I20251117 06:47:13.352939 171 agent_server.cpp:66] Register workload scheduler policy listener
I20251117 06:47:13.355458 171 cloud_backend_service.cpp:60] Doris CloudBackendService listening on 9060
I20251117 06:47:13.356564 171 thrift_server.cpp:423] ThriftServer 'backend' started on port: 9060
I20251117 06:47:13.406854 171 brpc_service.cpp:96] BRPC server bind to host: [0.0.0.0](http://0.0.0.0/), port: 8060
I20251117 06:47:13.434155 171 server.cpp:1144] Server[doris::CloudInternalServiceImpl] is serving on port=8060.
I20251117 06:47:13.434707 171 server.cpp:1147] Check out http://doris-cluster-test-compute-cg1-1:8060/ in web browser.
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_release/doris/be/src/common/signal_handler.h:421
1# 0x00007F7D44045520 in /lib/x86_64-linux-gnu/libc.so.6
2# pthread_kill in /lib/x86_64-linux-gnu/libc.so.6
3# raise in /lib/x86_64-linux-gnu/libc.so.6
4# abort in /lib/x86_64-linux-gnu/libc.so.6
5# __gnu_cxx::__verbose_terminate_handler() [clone .cold] at ../../../../libstdc++-v3/libsupc++/[vterminate.cc:75](http://vterminate.cc:75/)
6# __cxxabiv1::__terminate(void (*)()) at ../../../../libstdc++-v3/libsupc++/[eh_terminate.cc:48](http://eh_terminate.cc:48/)
7# 0x00005643819616D1 in /opt/apache-doris/be/lib/doris_be
8# 0x0000564381961824 in /opt/apache-doris/be/lib/doris_be
9# std::__throw_invalid_argument(char const*) at ../../../../../libstdc++-v3/src/c++11/[functexcept.cc:78](http://functexcept.cc:78/)
10# doris::io::FSFileCacheStorage::load_cache_info_into_memory(doris::io::BlockFileCache*) const at /home/zcp/repo_center/doris_release/doris/be/src/io/cache/fs_file_cache_storage.cpp:763
11# std::thread::_State_impl > >::_M_run() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211
12# execute_native_thread_routine at ../../../../../libstdc++-v3/src/c++11/[thread.cc:84](http://thread.cc:84/)
13# 0x00007F7D44097AC3 in /lib/x86_64-linux-gnu/libc.so.6
14# 0x00007F7D44129850 in /lib/x86_64-linux-gnu/libc.so.6
```

### What You Expected?

Resolving Doris be restart failure issues

### How to Reproduce?

The problem recurs every time you restart. You need to delete all files in the file_cache directory before you can restart normally.

### Anything Else?

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.