apache / apache/pulsar-client-python
Segfaults in Python 3.11 on UWSGI worker close
- 主要語言
- Python
- 星號
- 75
- 分支
- 53
- PR 合併指標
- 30 天內沒有已合併 PR
描述
We are using Python Pulsar producer clients in a web app that runs in UWSGI. We have our settings set to terminate and relaunch workers after a certain number of requests. We recently upgrade to Python 3.11 and the newest Pulsar client (3.3.0). At that point we started to see segfaults as the worker process was exiting. Here is the traceback that gets logged:
`!!! uWSGI process 85 got Segmentation Fault !!!
*** backtrace of 85 ***
uwsgi(uwsgi_backtrace+0x2f) [0x556ab933fcaf]
uwsgi(uwsgi_segfault+0x23) [0x556ab9340083]
/lib/x86_64-linux-gnu/libc.so.6(+0x3bfd0) [0x7f845c457fd0]
/usr/local/lib/python3.11/site-packages/pulsar_client.libs/libpulsar-2dce0d9f.so(+0x2b387a) [0x7f845770087a]
/usr/local/lib/python3.11/site-packages/_pulsar.cpython-311-x86_64-linux-gnu.so(+0xaf752) [0x7f8457f99752]
/usr/local/lib/python3.11/site-packages/_pulsar.cpython-311-x86_64-linux-gnu.so(+0x5399e) [0x7f8457f3d99e]
/usr/local/lib/python3.11/site-packages/_pulsar.cpython-311-x86_64-linux-gnu.so(+0x545ef) [0x7f8457f3e5ef]
/usr/local/lib/libpython3.11.so.1.0(+0x228f8a) [0x7f845c825f8a]
/usr/local/lib/libpython3.11.so.1.0(+0x1f4467) [0x7f845c7f1467]
/usr/local/lib/libpython3.11.so.1.0(+0x1f4030) [0x7f845c7f1030]
/usr/local/lib/libpython3.11.so.1.0(+0x29941f) [0x7f845c89641f]
/usr/local/lib/libpython3.11.so.1.0(+0x2993c5) [0x7f845c8963c5]
/usr/local/lib/libpython3.11.so.1.0(+0x1ed587) [0x7f845c7ea587]
/usr/local/lib/libpython3.11.so.1.0(_PyModule_ClearDict+0x115) [0x7f845c82bfe5]
/usr/local/lib/libpython3.11.so.1.0(+0x278bb6) [0x7f845c875bb6]
/usr/local/lib/libpython3.11.so.1.0(Py_FinalizeEx+0x12b) [0x7f845c8616ab]
uwsgi(uwsgi_plugins_atexit+0x71) [0x556ab933e121]
/lib/x86_64-linux-gnu/libc.so.6(+0x3e4dd) [0x7f845c45a4dd]
/lib/x86_64-linux-gnu/libc.so.6(+0x3e61a) [0x7f845c45a61a]
uwsgi(+0x3df6f) [0x556ab92f5f6f]
uwsgi(simple_goodbye_cruel_world+0x59) [0x556ab933f619]
uwsgi(+0x87658) [0x556ab933f658]
uwsgi(uwsgi_close_request+0x4d9) [0x556ab92f6bf9]
uwsgi(simple_loop_run+0xc0) [0x556ab933bbc0]
uwsgi(simple_loop+0x7b) [0x556ab933bc8b]
uwsgi(uwsgi_ignition+0x279) [0x556ab93403b9]
uwsgi(uwsgi_worker_run+0x25b) [0x556ab93449db]
uwsgi(uwsgi_run+0x43a) [0x556ab9344f2a]
uwsgi(+0x3aede) [0x556ab92f2ede]
/lib/x86_64-linux-gnu/libc.so.6(+0x271ca) [0x7f845c4431ca]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85) [0x7f845c443285]
uwsgi(_start+0x21) [0x556ab92f2f01]
*** end of backtrace ***`
The resulting core files are filling up the server disks. We could ulimit them but we'd prefer to avoid that.
貢獻指南
研究方向
首先,在 uWSGI 中啟用 worker 回收,重現 Python 3.11 與 Pulsar client 3.3.0 的組合,然後根據 backtrace 檢查涉及 _pulsar 和 libpulsar 的 shutdown 路徑。完成標準是 worker 關閉時不會發生 segmentation fault,且在正常回收期間不會產生 core 檔案。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- python
- 領域
- backend, distributed-systems
- Issue 類型
- 缺陷
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 停滯
- 描述清晰度
- 需要釐清
- 新手友好度
- 25/100