python / python/cpython

Data race: ctypes `PyCData_NewGetBuffer` reads `b_ptr` without the critical section `_ctypes_resize` holds

未關閉
#154,524 4 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

extension-modules topic-ctypes topic-free-threading type-crash
主要語言
Python
星號
77.2k
分支
36k
PR 合併指標
PR 指標待擷取

描述

Bug report

Bug description:

#131336 / #128182 made ctypes.resize and addressof/byref thread-safe by taking a critical section in _ctypes_resize, which reallocates obj->b_ptr:

https://github.com/python/cpython/blob/a2a846678e54b1b5defdccd451c573679094ff02/Modules/_ctypes/callproc.c#L1875-L1935

But the buffer-protocol getbuffer, PyCData_NewGetBuffer (reached via memoryview(cdata)), reads self->b_ptr without that critical section:

https://github.com/python/cpython/blob/a2a846678e54b1b5defdccd451c573679094ff02/Modules/_ctypes/_ctypes.c#L3103-L3130

So memoryview(obj) racing ctypes.resize(obj) reads a b_ptr that resize concurrently reallocates.

Reproducer:

import ctypes
from threading import Thread

buf = (ctypes.c_char * 64)()

def viewer():
    for _ in range(20000):
        try:
            memoryview(buf)
        except Exception:
            pass

def resizer():
    for i in range(20000):
        try:
            ctypes.resize(buf, 128 if (i & 1) else 256)
        except Exception:
            pass

threads  = [Thread(target=viewer)  for _ in range(6)]
threads += [Thread(target=resizer) for _ in range(2)]
for t in threads: t.start()
for t in threads: t.join()

TSAN Report:

==================
WARNING: ThreadSanitizer: data race (pid=1524546)
  Read of size 8 at 0x7fffb6b50208 by thread T1:
    #0 PyCData_NewGetBuffer /cpython/./Modules/_ctypes/_ctypes.c:3129:23 
    #1 PyObject_GetBuffer /cpython/Objects/abstract.c:455:15
    #2 _PyManagedBuffer_FromObject /cpython/Objects/memoryobject.c:97:9 
    #3 PyMemoryView_FromObjectAndFlags /cpython/Objects/memoryobject.c:813:42 
    #4 PyMemoryView_FromObject /cpython/Objects/memoryobject.c:856:12
    #5 memoryview_impl /cpython/Objects/memoryobject.c:1017:12 
    #6 memoryview /cpython/Objects/clinic/memoryobject.c.h:63:20 
    #7 type_call /cpython/Objects/typeobject.c:2472:11 
    #8 _PyObject_MakeTpCall /cpython/Objects/call.c:242:18
    #9 _PyObject_VectorcallTstate /cpython/./Include/internal/pycore_call.h:142:16)
    #10 PyObject_Vectorcall /cpython/Objects/call.c:327:12
    #11 _Py_VectorCall_StackRefSteal /cpython/Python/ceval.c:726:11 
    #12 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:4559:35 

  Previous write of size 8 at 0x7fffb6b50208 by thread T7:
    #0 _ctypes_resize_impl /cpython/./Modules/_ctypes/callproc.c
    #1 _ctypes_resize /cpython/./Modules/_ctypes/clinic/callproc.c.h:139:20
    #2 _Py_BuiltinCallFast_StackRef /cpython/Python/ceval.c:817:11
    #3 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:2510:35
    #4 _PyEval_EvalFrame /cpython/./Include/internal/pycore_ceval.h:122:16

SUMMARY: ThreadSanitizer: data race /cpython/./Modules/_ctypes/_ctypes.c:3129:23 in PyCData_NewGetBuffer
==================
CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs
  • gh-157759

貢獻指南

開啟貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

研究方向

從 Modules/_ctypes/_ctypes.c 中的 PyCData_NewGetBuffer 開始,並將其對 b_ptr 的存取與 Modules/_ctypes/callproc.c 中 _ctypes_resize_impl 的臨界區進行比較。使用 ThreadSanitizer 執行提供的多執行緒重現程式。當 memoryview/ctypes.resize 競態不再被回報,同時現有行為維持不變時,即表示完成。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
c, python
領域
backend
Issue 類型
缺陷
難度
4/5
預估耗時
3-5 天
活躍度
停滯
描述清晰度
描述清楚
新手友好度
30/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。