`os.sendfile` leaks header buffer exports on validation errors
Chưa có ai nhận issue này.
- Ngôn ngữ chính
- Python
- Star
- 77.2k
- Fork
- 35.9k
- Chỉ số merge pull request
- Chỉ số pull request đang chờ
Mô tả
Bug report
Bug description:
On macOS (and as far as I know FreeBSD), os.sendfile takes (among others) two headers and trailers parameters. Calling iov_setup acquires Py_buffer exports for headers, but several validation steps that come after can error and return before iov_cleanup is ever called.
The setup is done here:
The early returns after that never call iov_cleanup:
iov_cleanup is only called for trailers and headers after the syscall:
In practice, this can happen if trailers is not valid for example. The function (legitimately) raises a TypeError, but:
- There's a leak:
iov_setupallocated aniovecarray and aPy_bufferarray withPyMem_New. (A call toiov_cleanupwould free them, but it is not called.) - The input is left in an inconsistent state:
PyObject_GetBufferexported each header buffer. WithoutPyBuffer_Release, that export stays for the lifetime of the process, so for example abytearrayinheaderscan no longer be resized after the failure.
The same happens on macOS when the header-size overflow check returns after iov_setup, and when iov_setup for trailers fails after headers were already exported.
Reproducer
import os
import tempfile
header = bytearray(b"header")
with tempfile.TemporaryFile() as src, tempfile.TemporaryFile() as dst:
try:
os.sendfile(
dst.fileno(), src.fileno(), 0, 0,
headers=[header], trailers=object(),
)
except TypeError as exc:
print(exc) # sendfile() trailers must be a sequence
header.append(0)
# BufferError: Existing exports of data: object cannot be re-sized
The leak is properly reported by ASan/LSan:
=================================================================
==81993==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 80 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100fbbc00 in iov_setup posixmodule.c:12077
#2 0x000100fae4d0 in os_sendfile_impl posixmodule.c:12454
#3 0x000100fae4d0 in os_sendfile posixmodule.c.h:8233
#4 0x000100a6e23c in _PyObject_VectorcallTstate pycore_call.h:144
#5 0x000100a6e23c in PyObject_Vectorcall call.c:327
#6 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#7 0x000100ddb36c in _PyEval_EvalFrameDefault generated_cases.c.h:3325
#8 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#9 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#10 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#11 0x000100f19ea4 in run_mod pythonrun.c:1472
#12 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#13 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
#14 0x000100f7aad0 in pymain_run_command main.c:262
#15 0x000100f7aad0 in pymain_run_python main.c:706
#16 0x000100f7aad0 in Py_RunMain main.c:796
#17 0x000100f7b978 in pymain_main main.c:826
Direct leak of 16 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100fbbbc0 in iov_setup posixmodule.c:12071
#2 0x000100fae4d0 in os_sendfile_impl posixmodule.c:12454
#3 0x000100fae4d0 in os_sendfile posixmodule.c.h:8233
#4 0x000100a6e23c in _PyObject_VectorcallTstate pycore_call.h:144
#5 0x000100a6e23c in PyObject_Vectorcall call.c:327
#6 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#7 0x000100ddb36c in _PyEval_EvalFrameDefault generated_cases.c.h:3325
#8 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#9 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#10 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#11 0x000100f19ea4 in run_mod pythonrun.c:1472
#12 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#13 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
#14 0x000100f7aad0 in pymain_run_command main.c:262
#15 0x000100f7aad0 in pymain_run_python main.c:706
#16 0x000100f7aad0 in Py_RunMain main.c:796
#17 0x000100f7b978 in pymain_main main.c:826
Indirect leak of 64 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100bd3684 in _PyObject_MallocWithType pycore_object_alloc.h:46
#2 0x000100bd3684 in _PyType_AllocNoTrack typeobject.c:2523
#3 0x000100bd345c in PyType_GenericAlloc typeobject.c:2554
#4 0x000100bdd884 in type_call typeobject.c:2467
#5 0x000100a6cd90 in _PyObject_MakeTpCall call.c:242
#6 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#7 0x000100de4dc0 in _PyEval_EvalFrameDefault generated_cases.c.h:1846
#8 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#9 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#10 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#11 0x000100f19ea4 in run_mod pythonrun.c:1472
#12 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#13 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
#14 0x000100f7aad0 in pymain_run_command main.c:262
#15 0x000100f7aad0 in pymain_run_python main.c:706
#16 0x000100f7aad0 in Py_RunMain main.c:796
Indirect leak of 39 byte(s) in 1 object(s) allocated from:
#0 0x0001020e4e24 in malloc+0x70 (libclang_rt.asan_osx_dynamic.dylib:arm64+0x54e24)
#1 0x000100a5c9a4 in _PyBytes_FromSize bytesobject.c:121
#2 0x000100a5c9a4 in _PyBytes_Resize bytesobject.c:3356
#3 0x000100a3620c in bytearray_resize_lock_held bytearrayobject.c:280
#4 0x000100a37b50 in PyByteArray_Resize bytearrayobject.c:299
#5 0x000100a37b50 in bytearray___init___impl bytearrayobject.c:1021
#6 0x000100a37b50 in bytearray___init__ bytearrayobject.c.h:102
#7 0x000100bddab4 in type_call typeobject.c:2479
#8 0x000100a6cd90 in _PyObject_MakeTpCall call.c:242
#9 0x000100dc4e0c in _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775
#10 0x000100de4dc0 in _PyEval_EvalFrameDefault generated_cases.c.h:1846
#11 0x000100dc3ae0 in _PyEval_EvalFrame pycore_ceval.h:122
#12 0x000100dc3ae0 in _PyEval_Vector ceval.c:2156
#13 0x000100dc3ae0 in PyEval_EvalCode ceval.c:686
#14 0x000100f19ea4 in run_mod pythonrun.c:1472
#15 0x000100f1604c in _PyRun_StringFlagsWithName pythonrun.c:1260
#16 0x000100f15bd4 in _PyRun_SimpleStringFlagsWithName pythonrun.c:573
SUMMARY: AddressSanitizer: 199 byte(s) leaked in 4 allocation(s).
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
- gh-156288
Hướng dẫn đóng góp
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Hướng nghiên cứu
Bắt đầu trong Modules/posixmodule.c tại iov_setup và os_sendfile, lần theo các đường dẫn xác thực được xác định trong báo cáo. Sử dụng trình tái hiện macOS được cung cấp để xác minh rằng các lỗi xác thực giải phóng tài nguyên header và trailer, để các buffer vẫn có thể thay đổi kích thước và không làm rò rỉ các vùng cấp phát; PR được liên kết gh-156288 cho biết công việc đã được tiến hành.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- c, python
- Lĩnh vực
- operating-systems
- Loại issue
- Lỗi
- Độ khó
- 3/5
- Thời gian dự kiến
- 1-2 ngày
- Mức độ hoạt động
- Đình trệ
- Độ rõ ràng
- Đặc tả rõ ràng
- Mức phù hợp với người mới
- 25/100