Importing `readline` leaks memory on macOS
Chưa có ai nhận issue này.
- Ngôn ngữ chính
- Python
- Star
- 77.2k
- Fork
- 35.9k
- Chỉ số merge pull request
- Chỉ số pull request đang chờ
Mô tả
Bug report
Bug description:
In readline.c, setup_readline probes libedit's indexing behaviour by calling add_history twice (with the strings "1" and "2"), replacing one entry via replace_history_entry, and then calling clear_history to clean up.
On macOS, Apple's libedit clear_history does not free the line field (the internally strdup'd string) for all history entries, so one 2-byte allocation escapes. The replaced entry's old value is correctly freed by _py_free_history_entry_lock_held, but clear_history leaks one of the remaining strdup'd probe strings.
This leak (discovered while running unit tests for a Python package with LSan) is reported by Leak Sanitizer as:
Direct leak of 2 byte(s) in 1 object(s) allocated from:
#0 0x000101f3177c in strdup+0xfc (libclang_rt.asan_osx_dynamic.dylib:arm64+0x4d77c)
#1 0x0001dbc62ab0 (libedit.3.dylib:arm64e+0x14ab0)
#2 0x0001dbc63410 in history+0x6e4 (libedit.3.dylib:arm64e+0x15410)
#3 0x0001dbc5853c in add_history+0x50 (libedit.3.dylib:arm64e+0xa53c)
#4 0x00010b7b0ad4 in setup_readline readline.c:1373
#5 0x00010b7b0ad4 in PyInit_readline readline.c:1679
#6 0x000100ee935c in _PyImport_RunModInitFunc importdl.c:436
#7 0x000100ee36ac in import_run_extension import.c:2164
#8 0x000100ee6b90 in _imp_create_dynamic_impl import.c:5503
#9 0x000100ee6b90 in _imp_create_dynamic import.c.h:488
#10 0x000100aa5ffc in _PyVectorcall_Call call.c:273
#11 0x000100e01f68 in _PyEval_EvalFrameDefault generated_cases.c.h:2724
#12 0x000100dfc064 in _PyEval_EvalFrame pycore_ceval.h:122
#13 0x000100dfc064 in _PyEval_Vector ceval.c:2156
#14 0x000100aaa7c0 in _PyObject_VectorcallTstate pycore_call.h:144
#15 0x000100aaa7c0 in object_vacall call.c:823
#16 0x000100aaa29c in PyObject_CallMethodObjArgs call.c:960
#17 0x000100edf110 in import_find_and_load import.c:4125
#18 0x000100ede4a8 in PyImport_ImportModuleLevelObject import.c
#19 0x000100e37658 in _PyEval_ImportNameWithImport ceval.c:3011
#20 0x000100e37658 in _PyEval_ImportName ceval.c:2990
#21 0x000100e104ec in _PyEval_EvalFrameDefault generated_cases.c.h:6768
#22 0x000100af1004 in _PyEval_EvalFrame pycore_ceval.h:122
#23 0x000100af1004 in gen_send_ex2 genobject.c:280
#24 0x000100af1004 in gen_send_ex genobject.c:373
#25 0x000100aed464 in gen_iternext genobject.c:764
#26 0x000100df48cc in builtin_next bltinmodule.c:1764
#27 0x000100e14854 in _Py_BuiltinCallFast_StackRef ceval.c:824
#28 0x000100e14854 in _PyEval_EvalFrameDefault generated_cases.c.h:2420
#29 0x000100dfc064 in _PyEval_EvalFrame pycore_ceval.h:122
#30 0x000100dfc064 in _PyEval_Vector ceval.c:2156
#31 0x000100aa489c in _PyObject_VectorcallDictTstate call.c:146
#32 0x000100aa7130 in _PyObject_Call_Prepend call.c:504
#33 0x000100c292ec in call_method typeobject.c:3095
#34 0x000100c292ec in slot_tp_call typeobject.c:10913
#35 0x000100aa4d90 in _PyObject_MakeTpCall call.c:242
#36 0x000100dfce0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#37 0x000100e1336c in _PyEval_EvalFrameDefault generated_cases.c.h:3325
#38 0x000100dfc064 in _PyEval_EvalFrame pycore_ceval.h:122
#39 0x000100dfc064 in _PyEval_Vector ceval.c:2156
#40 0x000100aa9208 in _PyObject_VectorcallTstate pycore_call.h:144
#41 0x000100aa9208 in _PyObject_VectorcallPrepend call.c:877
SUMMARY: AddressSanitizer: 2 byte(s) leaked in 1 allocation(s).
That leak can easily by running the following under an LSan-enabled CPython on macOS.
ASAN_OPTIONS=detect_leaks=1 ./python.exe -c "import readline"
LLM-reviewing this issue surfaces two other issues around that code:
-
The probing block runs unconditionally even on GNU readline builds, where
libedit_history_startandlibedit_append_replace_history_offsetare never read (every use of those variables is guarded byif (using_libedit_emulation)). This meansadd_history/clear_historyare called unnecessarily on every readline import on Linux. -
There is no null check before calling
_py_free_history_entry_lock_held(old_entry)on the return value ofreplace_history_entry, which can returnNULLif the index is out of range.
Proposed fix
- Guard the entire probing block with
if (using_libedit_emulation). - Replace
clear_historyinside that block with an explicitremove_history(libedit_history_start)loop that calls_py_free_history_entry_lock_heldon each returned entry, bypassing the libedit bug. - Keep the unconditional
clear_historycall outside the block for readline state reset.
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
- gh-150537
Hướng dẫn đóng góp
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Hướng nghiên cứu
Bắt đầu trong Modules/readline.c tại setup_readline và xem xét block thăm dò libedit, việc dọn dẹp history của nó cũng như lời gọi clear_history không điều kiện. Tái hiện báo cáo bằng ASAN_OPTIONS=detect_leaks=1 ./python.exe -c "import readline" trên macOS; được coi là hoàn tất khi leak biến mất mà không thăm dò không cần thiết trong các bản build GNU readline và việc dọn dẹp history liên quan vẫn an toàn.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- c, python
- Lĩnh vực
- cli
- Loại issue
- Lỗi
- Độ khó
- 3/5
- Thời gian dự kiến
- 1-2 ngày
- Mức độ hoạt động
- Đình trệ
- Độ rõ ràng
- Đặc tả rõ ràng
- Mức phù hợp với người mới
- 25/100