`os.sendfile` leaks header buffer exports on validation errors
Dieses Issue hat noch niemand übernommen.
- Vorherrschende Sprache
- Python
- Sterne
- 77.2k
- Forks
- 35.9k
- PR-Merge-Kennzahlen
- PR-Kennzahlen ausstehend
Beschreibung
Bug report
Bug description:
On macOS (and as far as I know FreeBSD), os.sendfile takes (among others) two headers and trailers parameters. Calling iov_setup acquires Py_buffer exports for headers, but several validation steps that come after can error and return before iov_cleanup is ever called.
The setup is done here:
The early returns after that never call iov_cleanup:
iov_cleanup is only called for trailers and headers after the syscall:
In practice, this can happen if trailers is not valid for example. The function (legitimately) raises a TypeError, but:
- There's a leak:
iov_setupallocated aniovecarray and aPy_bufferarray withPyMem_New. (A call toiov_cleanupwould free them, but it is not called.) - The input is left in an inconsistent state:
PyObject_GetBufferexported each header buffer. WithoutPyBuffer_Release, that export stays for the lifetime of the process, so for example abytearrayinheaderscan no longer be resized after the failure.
The same happens on macOS when the header-size overflow check returns after iov_setup, and when iov_setup for trailers fails after headers were already exported.
Reproducer
import os
import tempfile
header = bytearray(b"header")
with tempfile.TemporaryFile() as src, tempfile.TemporaryFile() as dst:
try:
os.sendfile(
dst.fileno(), src.fileno(), 0, 0,
headers=[header], trailers=object(),
)
except TypeError as exc:
print(exc) # sendfile() trailers must be a sequence
header.append(0)
# BufferError: Existing exports of data: object cannot be re-sized
The leak is properly reported by ASan/LSan:
=================================================================
==81993==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 80 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100fbbc00 in iov_setup posixmodule.c:12077
#2 0x000100fae4d0 in os_sendfile_impl posixmodule.c:12454
#3 0x000100fae4d0 in os_sendfile posixmodule.c.h:8233
#4 0x000100a6e23c in _PyObject_VectorcallTstate pycore_call.h:144
#5 0x000100a6e23c in PyObject_Vectorcall call.c:327
#6 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#7 0x000100ddb36c in _PyEval_EvalFrameDefault generated_cases.c.h:3325
#8 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#9 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#10 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#11 0x000100f19ea4 in run_mod pythonrun.c:1472
#12 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#13 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
#14 0x000100f7aad0 in pymain_run_command main.c:262
#15 0x000100f7aad0 in pymain_run_python main.c:706
#16 0x000100f7aad0 in Py_RunMain main.c:796
#17 0x000100f7b978 in pymain_main main.c:826
Direct leak of 16 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100fbbbc0 in iov_setup posixmodule.c:12071
#2 0x000100fae4d0 in os_sendfile_impl posixmodule.c:12454
#3 0x000100fae4d0 in os_sendfile posixmodule.c.h:8233
#4 0x000100a6e23c in _PyObject_VectorcallTstate pycore_call.h:144
#5 0x000100a6e23c in PyObject_Vectorcall call.c:327
#6 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#7 0x000100ddb36c in _PyEval_EvalFrameDefault generated_cases.c.h:3325
#8 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#9 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#10 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#11 0x000100f19ea4 in run_mod pythonrun.c:1472
#12 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#13 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
#14 0x000100f7aad0 in pymain_run_command main.c:262
#15 0x000100f7aad0 in pymain_run_python main.c:706
#16 0x000100f7aad0 in Py_RunMain main.c:796
#17 0x000100f7b978 in pymain_main main.c:826
Indirect leak of 64 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100bd3684 in _PyObject_MallocWithType pycore_object_alloc.h:46
#2 0x000100bd3684 in _PyType_AllocNoTrack typeobject.c:2523
#3 0x000100bd345c in PyType_GenericAlloc typeobject.c:2554
#4 0x000100bdd884 in type_call typeobject.c:2467
#5 0x000100a6cd90 in _PyObject_MakeTpCall call.c:242
#6 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#7 0x000100de4dc0 in _PyEval_EvalFrameDefault generated_cases.c.h:1846
#8 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#9 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#10 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#11 0x000100f19ea4 in run_mod pythonrun.c:1472
#12 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#13 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
#14 0x000100f7aad0 in pymain_run_command main.c:262
#15 0x000100f7aad0 in pymain_run_python main.c:706
#16 0x000100f7aad0 in Py_RunMain main.c:796
Indirect leak of 39 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100a5c9a4 in _PyBytes_FromSize bytesobject.c:121
#2 0x000100a5c9a4 in _PyBytes_Resize bytesobject.c:3356
#3 0x000100a3620c in bytearray_resize_lock_held bytearrayobject.c:280
#4 0x000100a37b50 in PyByteArray_Resize bytearrayobject.c:299
#5 0x000100a37b50 in bytearray___init___impl bytearrayobject.c:1021
#6 0x000100a37b50 in bytearray___init__ bytearrayobject.c.h:102
#7 0x000100bddab4 in type_call typeobject.c:2479
#8 0x000100a6cd90 in _PyObject_MakeTpCall call.c:242
#9 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#10 0x000100de4dc0 in _PyEval_EvalFrameDefault generated_cases.c.h:1846
#11 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#12 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#13 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#14 0x000100f19ea4 in run_mod pythonrun.c:1472
#15 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#16 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
SUMMARY: AddressSanitizer: 199 byte(s) leaked in 4 allocation(s).
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
- gh-156288
Beitragsleitfaden
Erste Schritte
- Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
- Forke das Repository und arbeite in einem Branch.
- Öffne einen Pull Request, der die Issue-Nummer nennt.
Rechercherichtung
Beginne in Modules/posixmodule.c bei iov_setup und os_sendfile und folge den im Bericht identifizierten Validierungspfaden. Verwende den bereitgestellten macOS-Reproducer, um zu überprüfen, dass Validierungsfehler Ressourcen von Header und Trailer freigeben, Puffer veränderbar lassen und keine Allokationen leaken; der verknüpfte PR gh-156288 zeigt an, dass die Arbeit bereits im Gange ist.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- c, python
- Bereich
- operating-systems
- Issue-Typ
- Bug
- Schwierigkeit
- 3/5
- Geschätzter Aufwand
- 1-2 Tage
- Aktivitätsstatus
- Veraltet
- Klarheit
- Klar beschrieben
- Anfängerfreundlichkeit
- 25/100