求助:卡死问题排查
- Dominant language
- C++
- Stars
- 17.6k
- Forks
- 4.1k
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 69
Description
**Describe the bug (描述bug)**
adx服务在启动的时候大概率会出现卡死。vars接口显示创建了大量bthread和channel_connections,并且不断增长,最后导致OOM。如果能成功启动,服务运行还是比较稳定,高峰时bthread最多1800个,channel_connections最多2500个。
brpc服务接收到请求后会广播给下游dsp。协议基本都是http。我们对每一个dsp请求都会启一个bthread,bthread里同步请求下游服务,设定的超时时间为500ms左右。处理服务请求的bthread会join每个dsp的bthread。
服务最高并发不到100。dsp120个左右。
gdb调试的时候,worker线程堆栈目前看到有两种:
1. worker都在发送dsp请求,暂停在dsp的CallMethod。看堆栈貌似把pthread卡住了

2. worker都在建立ssl链接

有没有排查思路?
**To Reproduce (复现方法)**
**Expected behavior (期望行为)**
**Versions (各种版本)**
OS: ubuntu 22.04.3
Compiler: g++ 11.4
brpc: 1.6
protobuf: 3.20.1
**Additional context/screenshots (更多上下文/截图)**
Contributor guide
Research direction
Start with the vars output and the two GDB worker-stack patterns described: blocked DSP CallMethod calls and SSL connection setup. Inspect how the service creates and joins bthreads for each downstream DSP request, and how channel_connections grow during startup. Done means identifying the cause of the unbounded growth or documenting a reproducible diagnosis and concrete mitigation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, distributed-systems, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100