`os.sendfile` leaks header buffer exports on validation errors
Personne n'a encore pris cette issue.
- Langage dominant
- Python
- Étoiles
- 77.2k
- Forks
- 35.9k
- Métriques de merge des PR
- Métriques de PR en attente
Description
Bug report
Bug description:
On macOS (and as far as I know FreeBSD), os.sendfile takes (among others) two headers and trailers parameters. Calling iov_setup acquires Py_buffer exports for headers, but several validation steps that come after can error and return before iov_cleanup is ever called.
The setup is done here:
The early returns after that never call iov_cleanup:
iov_cleanup is only called for trailers and headers after the syscall:
In practice, this can happen if trailers is not valid for example. The function (legitimately) raises a TypeError, but:
- There's a leak:
iov_setupallocated aniovecarray and aPy_bufferarray withPyMem_New. (A call toiov_cleanupwould free them, but it is not called.) - The input is left in an inconsistent state:
PyObject_GetBufferexported each header buffer. WithoutPyBuffer_Release, that export stays for the lifetime of the process, so for example abytearrayinheaderscan no longer be resized after the failure.
The same happens on macOS when the header-size overflow check returns after iov_setup, and when iov_setup for trailers fails after headers were already exported.
Reproducer
import os
import tempfile
header = bytearray(b"header")
with tempfile.TemporaryFile() as src, tempfile.TemporaryFile() as dst:
try:
os.sendfile(
dst.fileno(), src.fileno(), 0, 0,
headers=[header], trailers=object(),
)
except TypeError as exc:
print(exc) # sendfile() trailers must be a sequence
header.append(0)
# BufferError: Existing exports of data: object cannot be re-sized
The leak is properly reported by ASan/LSan:
=================================================================
==81993==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 80 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100fbbc00 in iov_setup posixmodule.c:12077
#2 0x000100fae4d0 in os_sendfile_impl posixmodule.c:12454
#3 0x000100fae4d0 in os_sendfile posixmodule.c.h:8233
#4 0x000100a6e23c in _PyObject_VectorcallTstate pycore_call.h:144
#5 0x000100a6e23c in PyObject_Vectorcall call.c:327
#6 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#7 0x000100ddb36c in _PyEval_EvalFrameDefault generated_cases.c.h:3325
#8 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#9 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#10 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#11 0x000100f19ea4 in run_mod pythonrun.c:1472
#12 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#13 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
#14 0x000100f7aad0 in pymain_run_command main.c:262
#15 0x000100f7aad0 in pymain_run_python main.c:706
#16 0x000100f7aad0 in Py_RunMain main.c:796
#17 0x000100f7b978 in pymain_main main.c:826
Direct leak of 16 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100fbbbc0 in iov_setup posixmodule.c:12071
#2 0x000100fae4d0 in os_sendfile_impl posixmodule.c:12454
#3 0x000100fae4d0 in os_sendfile posixmodule.c.h:8233
#4 0x000100a6e23c in _PyObject_VectorcallTstate pycore_call.h:144
#5 0x000100a6e23c in PyObject_Vectorcall call.c:327
#6 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#7 0x000100ddb36c in _PyEval_EvalFrameDefault generated_cases.c.h:3325
#8 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#9 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#10 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#11 0x000100f19ea4 in run_mod pythonrun.c:1472
#12 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#13 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
#14 0x000100f7aad0 in pymain_run_command main.c:262
#15 0x000100f7aad0 in pymain_run_python main.c:706
#16 0x000100f7aad0 in Py_RunMain main.c:796
#17 0x000100f7b978 in pymain_main main.c:826
Indirect leak of 64 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100bd3684 in _PyObject_MallocWithType pycore_object_alloc.h:46
#2 0x000100bd3684 in _PyType_AllocNoTrack typeobject.c:2523
#3 0x000100bd345c in PyType_GenericAlloc typeobject.c:2554
#4 0x000100bdd884 in type_call typeobject.c:2467
#5 0x000100a6cd90 in _PyObject_MakeTpCall call.c:242
#6 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#7 0x000100de4dc0 in _PyEval_EvalFrameDefault generated_cases.c.h:1846
#8 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#9 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#10 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#11 0x000100f19ea4 in run_mod pythonrun.c:1472
#12 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#13 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
#14 0x000100f7aad0 in pymain_run_command main.c:262
#15 0x000100f7aad0 in pymain_run_python main.c:706
#16 0x000100f7aad0 in Py_RunMain main.c:796
Indirect leak of 39 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100a5c9a4 in _PyBytes_FromSize bytesobject.c:121
#2 0x000100a5c9a4 in _PyBytes_Resize bytesobject.c:3356
#3 0x000100a3620c in bytearray_resize_lock_held bytearrayobject.c:280
#4 0x000100a37b50 in PyByteArray_Resize bytearrayobject.c:299
#5 0x000100a37b50 in bytearray___init___impl bytearrayobject.c:1021
#6 0x000100a37b50 in bytearray___init__ bytearrayobject.c.h:102
#7 0x000100bddab4 in type_call typeobject.c:2479
#8 0x000100a6cd90 in _PyObject_MakeTpCall call.c:242
#9 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#10 0x000100de4dc0 in _PyEval_EvalFrameDefault generated_cases.c.h:1846
#11 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#12 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#13 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#14 0x000100f19ea4 in run_mod pythonrun.c:1472
#15 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#16 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
SUMMARY: AddressSanitizer: 199 byte(s) leaked in 4 allocation(s).
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
- gh-156288
Guide de contribution
Ouvrir le guide de contribution
Par où commencer
- Lisez l'issue en entier, puis le guide de contribution du projet.
- Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
- Forkez le dépôt et travaillez sur une branche.
- Ouvrez une pull request qui référence le numéro de l'issue.
Piste de recherche
Commencez dans Modules/posixmodule.c, au niveau de iov_setup et os_sendfile, en suivant les chemins de validation identifiés dans le rapport. Utilisez le reproducer macOS fourni pour vérifier que les échecs de validation libèrent les ressources du header et du trailer, laissent les buffers redimensionnables et n’entraînent aucune fuite d’allocations ; la PR liée gh-156288 indique que le travail est déjà en cours.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- c, python
- Domaine
- operating-systems
- Type d'issue
- Bug
- Difficulté
- 3/5
- Temps estimé
- 1-2 jours
- Activité
- À l'abandon
- Clarté
- Clairement spécifiée
- Accessibilité débutants
- 25/100