Importing `readline` leaks memory on macOS
Nessuno ha ancora preso questa issue.
- Lingua principale
- Python
- Stelle
- 77.2k
- Fork
- 35.9k
- Metriche di merge delle PR
- Metriche PR in attesa
Descrizione
Bug report
Bug description:
In readline.c, setup_readline probes libedit's indexing behaviour by calling add_history twice (with the strings "1" and "2"), replacing one entry via replace_history_entry, and then calling clear_history to clean up.
On macOS, Apple's libedit clear_history does not free the line field (the internally strdup'd string) for all history entries, so one 2-byte allocation escapes. The replaced entry's old value is correctly freed by _py_free_history_entry_lock_held, but clear_history leaks one of the remaining strdup'd probe strings.
This leak (discovered while running unit tests for a Python package with LSan) is reported by Leak Sanitizer as:
Direct leak of 2 byte(s) in 1 object(s) allocated from:
#0 0x000101f3177c in strdup+0xfc (libclang_rt.asan_osx_dynamic.dylib:arm64+0x4d77c)
#1 0x0001dbc62ab0 (libedit.3.dylib:arm64e+0x14ab0)
#2 0x0001dbc63410 in history+0x6e4 (libedit.3.dylib:arm64e+0x15410)
#3 0x0001dbc5853c in add_history+0x50 (libedit.3.dylib:arm64e+0xa53c)
#4 0x00010b7b0ad4 in setup_readline readline.c:1373
#5 0x00010b7b0ad4 in PyInit_readline readline.c:1679
#6 0x000100ee935c in _PyImport_RunModInitFunc importdl.c:436
#7 0x000100ee36ac in import_run_extension import.c:2164
#8 0x000100ee6b90 in _imp_create_dynamic_impl import.c:5503
#9 0x000100ee6b90 in _imp_create_dynamic import.c.h:488
#10 0x000100aa5ffc in _PyVectorcall_Call call.c:273
#11 0x000100e01f68 in _PyEval_EvalFrameDefault generated_cases.c.h:2724
#12 0x000100dfc064 in _PyEval_EvalFrame pycore_ceval.h:122
#13 0x000100dfc064 in _PyEval_Vector ceval.c:2156
#14 0x000100aaa7c0 in _PyObject_VectorcallTstate pycore_call.h:144
#15 0x000100aaa7c0 in object_vacall call.c:823
#16 0x000100aaa29c in PyObject_CallMethodObjArgs call.c:960
#17 0x000100edf110 in import_find_and_load import.c:4125
#18 0x000100ede4a8 in PyImport_ImportModuleLevelObject import.c
#19 0x000100e37658 in _PyEval_ImportNameWithImport ceval.c:3011
#20 0x000100e37658 in _PyEval_ImportName ceval.c:2990
#21 0x000100e104ec in _PyEval_EvalFrameDefault generated_cases.c.h:6768
#22 0x000100af1004 in _PyEval_EvalFrame pycore_ceval.h:122
#23 0x000100af1004 in gen_send_ex2 genobject.c:280
#24 0x000100af1004 in gen_send_ex genobject.c:373
#25 0x000100aed464 in gen_iternext genobject.c:764
#26 0x000100df48cc in builtin_next bltinmodule.c:1764
#27 0x000100e14854 in _Py_BuiltinCallFast_StackRef ceval.c:824
#28 0x000100e14854 in _PyEval_EvalFrameDefault generated_cases.c.h:2420
#29 0x000100dfc064 in _PyEval_EvalFrame pycore_ceval.h:122
#30 0x000100dfc064 in _PyEval_Vector ceval.c:2156
#31 0x000100aa489c in _PyObject_VectorcallDictTstate call.c:146
#32 0x000100aa7130 in _PyObject_Call_Prepend call.c:504
#33 0x000100c292ec in call_method typeobject.c:3095
#34 0x000100c292ec in slot_tp_call typeobject.c:10913
#35 0x000100aa4d90 in _PyObject_MakeTpCall call.c:242
#36 0x000100dfce0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#37 0x000100e1336c in _PyEval_EvalFrameDefault generated_cases.c.h:3325
#38 0x000100dfc064 in _PyEval_EvalFrame pycore_ceval.h:122
#39 0x000100dfc064 in _PyEval_Vector ceval.c:2156
#40 0x000100aa9208 in _PyObject_VectorcallTstate pycore_call.h:144
#41 0x000100aa9208 in _PyObject_VectorcallPrepend call.c:877
SUMMARY: AddressSanitizer: 2 byte(s) leaked in 1 allocation(s).
That leak can easily by running the following under an LSan-enabled CPython on macOS.
ASAN_OPTIONS=detect_leaks=1 ./python.exe -c "import readline"
LLM-reviewing this issue surfaces two other issues around that code:
-
The probing block runs unconditionally even on GNU readline builds, where
libedit_history_startandlibedit_append_replace_history_offsetare never read (every use of those variables is guarded byif (using_libedit_emulation)). This meansadd_history/clear_historyare called unnecessarily on every readline import on Linux. -
There is no null check before calling
_py_free_history_entry_lock_held(old_entry)on the return value ofreplace_history_entry, which can returnNULLif the index is out of range.
Proposed fix
- Guard the entire probing block with
if (using_libedit_emulation). - Replace
clear_historyinside that block with an explicitremove_history(libedit_history_start)loop that calls_py_free_history_entry_lock_heldon each returned entry, bypassing the libedit bug. - Keep the unconditional
clear_historycall outside the block for readline state reset.
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
- gh-150537
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Direzione di ricerca
Inizia in Modules/readline.c, nella funzione setup_readline, e rivedi il blocco di rilevamento di libedit, la pulizia della cronologia e la chiamata incondizionata a clear_history. Riproduci il report con ASAN_OPTIONS=detect_leaks=1 ./python.exe -c "import readline" su macOS; il lavoro è completato quando il leak è scomparso senza rilevamenti non necessari nelle build di GNU readline e la pulizia della cronologia pertinente rimane sicura.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- c, python
- Ambito
- cli
- Tipo di issue
- Bug
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Stato di attività
- Ferma
- Chiarezza
- Specificata chiaramente
- Idoneità per principianti
- 25/100