Importing `readline` leaks memory on macOS
还没有人认领这个 Issue。
- 主要语言
- Python
- 星标
- 77.2k
- 派生
- 35.9k
- PR 合并指标
- PR 指标待抓取
描述
Bug report
Bug description:
In readline.c, setup_readline probes libedit's indexing behaviour by calling add_history twice (with the strings "1" and "2"), replacing one entry via replace_history_entry, and then calling clear_history to clean up.
On macOS, Apple's libedit clear_history does not free the line field (the internally strdup'd string) for all history entries, so one 2-byte allocation escapes. The replaced entry's old value is correctly freed by _py_free_history_entry_lock_held, but clear_history leaks one of the remaining strdup'd probe strings.
This leak (discovered while running unit tests for a Python package with LSan) is reported by Leak Sanitizer as:
Direct leak of 2 byte(s) in 1 object(s) allocated from:
#0 0x000101f3177c in strdup+0xfc (libclang_rt.asan_osx_dynamic.dylib:arm64+0x4d77c)
#1 0x0001dbc62ab0 (libedit.3.dylib:arm64e+0x14ab0)
#2 0x0001dbc63410 in history+0x6e4 (libedit.3.dylib:arm64e+0x15410)
#3 0x0001dbc5853c in add_history+0x50 (libedit.3.dylib:arm64e+0xa53c)
#4 0x00010b7b0ad4 in setup_readline readline.c:1373
#5 0x00010b7b0ad4 in PyInit_readline readline.c:1679
#6 0x000100ee935c in _PyImport_RunModInitFunc importdl.c:436
#7 0x000100ee36ac in import_run_extension import.c:2164
#8 0x000100ee6b90 in _imp_create_dynamic_impl import.c:5503
#9 0x000100ee6b90 in _imp_create_dynamic import.c.h:488
#10 0x000100aa5ffc in _PyVectorcall_Call call.c:273
#11 0x000100e01f68 in _PyEval_EvalFrameDefault generated_cases.c.h:2724
#12 0x000100dfc064 in _PyEval_EvalFrame pycore_ceval.h:122
#13 0x000100dfc064 in _PyEval_Vector ceval.c:2156
#14 0x000100aaa7c0 in _PyObject_VectorcallTstate pycore_call.h:144
#15 0x000100aaa7c0 in object_vacall call.c:823
#16 0x000100aaa29c in PyObject_CallMethodObjArgs call.c:960
#17 0x000100edf110 in import_find_and_load import.c:4125
#18 0x000100ede4a8 in PyImport_ImportModuleLevelObject import.c
#19 0x000100e37658 in _PyEval_ImportNameWithImport ceval.c:3011
#20 0x000100e37658 in _PyEval_ImportName ceval.c:2990
#21 0x000100e104ec in _PyEval_EvalFrameDefault generated_cases.c.h:6768
#22 0x000100af1004 in _PyEval_EvalFrame pycore_ceval.h:122
#23 0x000100af1004 in gen_send_ex2 genobject.c:280
#24 0x000100af1004 in gen_send_ex genobject.c:373
#25 0x000100aed464 in gen_iternext genobject.c:764
#26 0x000100df48cc in builtin_next bltinmodule.c:1764
#27 0x000100e14854 in _Py_BuiltinCallFast_StackRef ceval.c:824
#28 0x000100e14854 in _PyEval_EvalFrameDefault generated_cases.c.h:2420
#29 0x000100dfc064 in _PyEval_EvalFrame pycore_ceval.h:122
#30 0x000100dfc064 in _PyEval_Vector ceval.c:2156
#31 0x000100aa489c in _PyObject_VectorcallDictTstate call.c:146
#32 0x000100aa7130 in _PyObject_Call_Prepend call.c:504
#33 0x000100c292ec in call_method typeobject.c:3095
#34 0x000100c292ec in slot_tp_call typeobject.c:10913
#35 0x000100aa4d90 in _PyObject_MakeTpCall call.c:242
#36 0x000100dfce0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#37 0x000100e1336c in _PyEval_EvalFrameDefault generated_cases.c.h:3325
#38 0x000100dfc064 in _PyEval_EvalFrame pycore_ceval.h:122
#39 0x000100dfc064 in _PyEval_Vector ceval.c:2156
#40 0x000100aa9208 in _PyObject_VectorcallTstate pycore_call.h:144
#41 0x000100aa9208 in _PyObject_VectorcallPrepend call.c:877
SUMMARY: AddressSanitizer: 2 byte(s) leaked in 1 allocation(s).
That leak can easily by running the following under an LSan-enabled CPython on macOS.
ASAN_OPTIONS=detect_leaks=1 ./python.exe -c "import readline"
LLM-reviewing this issue surfaces two other issues around that code:
-
The probing block runs unconditionally even on GNU readline builds, where
libedit_history_startandlibedit_append_replace_history_offsetare never read (every use of those variables is guarded byif (using_libedit_emulation)). This meansadd_history/clear_historyare called unnecessarily on every readline import on Linux. -
There is no null check before calling
_py_free_history_entry_lock_held(old_entry)on the return value ofreplace_history_entry, which can returnNULLif the index is out of range.
Proposed fix
- Guard the entire probing block with
if (using_libedit_emulation). - Replace
clear_historyinside that block with an explicitremove_history(libedit_history_start)loop that calls_py_free_history_entry_lock_heldon each returned entry, bypassing the libedit bug. - Keep the unconditional
clear_historycall outside the block for readline state reset.
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
- gh-150537
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
从 Modules/readline.c 中的 setup_readline 开始,检查 libedit 探测代码块、其历史记录清理,以及无条件调用 clear_history 的部分。在 macOS 上使用 ASAN_OPTIONS=detect_leaks=1 ./python.exe -c "import readline" 重现该报告;当 GNU readline 构建中无需进行不必要的探测即可消除泄漏,并且相关的历史记录清理仍然安全时,即视为完成。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- c, python
- 领域
- cli
- Issue 类型
- 缺陷
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 停滞
- 描述清晰度
- 描述清楚
- 新手友好度
- 25/100