`os.sendfile` leaks header buffer exports on validation errors
还没有人认领这个 Issue。
- 主要语言
- Python
- 星标
- 77.2k
- 派生
- 35.9k
- PR 合并指标
- PR 指标待抓取
描述
Bug report
Bug description:
On macOS (and as far as I know FreeBSD), os.sendfile takes (among others) two headers and trailers parameters. Calling iov_setup acquires Py_buffer exports for headers, but several validation steps that come after can error and return before iov_cleanup is ever called.
The setup is done here:
The early returns after that never call iov_cleanup:
iov_cleanup is only called for trailers and headers after the syscall:
In practice, this can happen if trailers is not valid for example. The function (legitimately) raises a TypeError, but:
- There's a leak:
iov_setupallocated aniovecarray and aPy_bufferarray withPyMem_New. (A call toiov_cleanupwould free them, but it is not called.) - The input is left in an inconsistent state:
PyObject_GetBufferexported each header buffer. WithoutPyBuffer_Release, that export stays for the lifetime of the process, so for example abytearrayinheaderscan no longer be resized after the failure.
The same happens on macOS when the header-size overflow check returns after iov_setup, and when iov_setup for trailers fails after headers were already exported.
Reproducer
import os
import tempfile
header = bytearray(b"header")
with tempfile.TemporaryFile() as src, tempfile.TemporaryFile() as dst:
try:
os.sendfile(
dst.fileno(), src.fileno(), 0, 0,
headers=[header], trailers=object(),
)
except TypeError as exc:
print(exc) # sendfile() trailers must be a sequence
header.append(0)
# BufferError: Existing exports of data: object cannot be re-sized
The leak is properly reported by ASan/LSan:
=================================================================
==81993==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 80 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100fbbc00 in iov_setup posixmodule.c:12077
#2 0x000100fae4d0 in os_sendfile_impl posixmodule.c:12454
#3 0x000100fae4d0 in os_sendfile posixmodule.c.h:8233
#4 0x000100a6e23c in _PyObject_VectorcallTstate pycore_call.h:144
#5 0x000100a6e23c in PyObject_Vectorcall call.c:327
#6 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#7 0x000100ddb36c in _PyEval_EvalFrameDefault generated_cases.c.h:3325
#8 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#9 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#10 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#11 0x000100f19ea4 in run_mod pythonrun.c:1472
#12 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#13 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
#14 0x000100f7aad0 in pymain_run_command main.c:262
#15 0x000100f7aad0 in pymain_run_python main.c:706
#16 0x000100f7aad0 in Py_RunMain main.c:796
#17 0x000100f7b978 in pymain_main main.c:826
Direct leak of 16 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100fbbbc0 in iov_setup posixmodule.c:12071
#2 0x000100fae4d0 in os_sendfile_impl posixmodule.c:12454
#3 0x000100fae4d0 in os_sendfile posixmodule.c.h:8233
#4 0x000100a6e23c in _PyObject_VectorcallTstate pycore_call.h:144
#5 0x000100a6e23c in PyObject_Vectorcall call.c:327
#6 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#7 0x000100ddb36c in _PyEval_EvalFrameDefault generated_cases.c.h:3325
#8 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#9 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#10 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#11 0x000100f19ea4 in run_mod pythonrun.c:1472
#12 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#13 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
#14 0x000100f7aad0 in pymain_run_command main.c:262
#15 0x000100f7aad0 in pymain_run_python main.c:706
#16 0x000100f7aad0 in Py_RunMain main.c:796
#17 0x000100f7b978 in pymain_main main.c:826
Indirect leak of 64 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100bd3684 in _PyObject_MallocWithType pycore_object_alloc.h:46
#2 0x000100bd3684 in _PyType_AllocNoTrack typeobject.c:2523
#3 0x000100bd345c in PyType_GenericAlloc typeobject.c:2554
#4 0x000100bdd884 in type_call typeobject.c:2467
#5 0x000100a6cd90 in _PyObject_MakeTpCall call.c:242
#6 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#7 0x000100de4dc0 in _PyEval_EvalFrameDefault generated_cases.c.h:1846
#8 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#9 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#10 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#11 0x000100f19ea4 in run_mod pythonrun.c:1472
#12 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#13 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
#14 0x000100f7aad0 in pymain_run_command main.c:262
#15 0x000100f7aad0 in pymain_run_python main.c:706
#16 0x000100f7aad0 in Py_RunMain main.c:796
Indirect leak of 39 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100a5c9a4 in _PyBytes_FromSize bytesobject.c:121
#2 0x000100a5c9a4 in _PyBytes_Resize bytesobject.c:3356
#3 0x000100a3620c in bytearray_resize_lock_held bytearrayobject.c:280
#4 0x000100a37b50 in PyByteArray_Resize bytearrayobject.c:299
#5 0x000100a37b50 in bytearray___init___impl bytearrayobject.c:1021
#6 0x000100a37b50 in bytearray___init__ bytearrayobject.c.h:102
#7 0x000100bddab4 in type_call typeobject.c:2479
#8 0x000100a6cd90 in _PyObject_MakeTpCall call.c:242
#9 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#10 0x000100de4dc0 in _PyEval_EvalFrameDefault generated_cases.c.h:1846
#11 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#12 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#13 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#14 0x000100f19ea4 in run_mod pythonrun.c:1472
#15 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#16 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
SUMMARY: AddressSanitizer: 199 byte(s) leaked in 4 allocation(s).
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
- gh-156288
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
从 Modules/posixmodule.c 中的 iov_setup 和 os_sendfile 开始,跟踪报告中确定的验证路径。使用提供的 macOS 复现程序,验证验证失败会释放 header 和 trailer 资源,使缓冲区保持可调整大小,并且不会泄漏分配;关联的 PR gh-156288 表明相关工作已经在进行中。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- c, python
- 领域
- operating-systems
- Issue 类型
- 缺陷
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 停滞
- 描述清晰度
- 描述清楚
- 新手友好度
- 25/100