python / python/cpython

Data race: ctypes `PyCData_NewGetBuffer` reads `b_ptr` without the critical section `_ctypes_resize` holds

未关闭
#154,524 4 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

extension-modules topic-ctypes topic-free-threading type-crash
主要语言
Python
星标
77.2k
派生
35.9k
PR 合并指标
PR 指标待抓取

描述

Bug report

Bug description:

#131336 / #128182 made ctypes.resize and addressof/byref thread-safe by taking a critical section in _ctypes_resize, which reallocates obj->b_ptr:

https://github.com/python/cpython/blob/a2a846678e54b1b5defdccd451c573679094ff02/Modules/_ctypes/callproc.c#L1875-L1935

But the buffer-protocol getbuffer, PyCData_NewGetBuffer (reached via memoryview(cdata)), reads self->b_ptr without that critical section:

https://github.com/python/cpython/blob/a2a846678e54b1b5defdccd451c573679094ff02/Modules/_ctypes/_ctypes.c#L3103-L3130

So memoryview(obj) racing ctypes.resize(obj) reads a b_ptr that resize concurrently reallocates.

Reproducer:

import ctypes
from threading import Thread

buf = (ctypes.c_char * 64)()

def viewer():
    for _ in range(20000):
        try:
            memoryview(buf)
        except Exception:
            pass

def resizer():
    for i in range(20000):
        try:
            ctypes.resize(buf, 128 if (i & 1) else 256)
        except Exception:
            pass

threads  = [Thread(target=viewer)  for _ in range(6)]
threads += [Thread(target=resizer) for _ in range(2)]
for t in threads: t.start()
for t in threads: t.join()

TSAN Report:

==================
WARNING: ThreadSanitizer: data race (pid=1524546)
  Read of size 8 at 0x7fffb6b50208 by thread T1:
    #0 PyCData_NewGetBuffer /cpython/./Modules/_ctypes/_ctypes.c:3129:23 
    #1 PyObject_GetBuffer /cpython/Objects/abstract.c:455:15
    #2 _PyManagedBuffer_FromObject /cpython/Objects/memoryobject.c:97:9 
    #3 PyMemoryView_FromObjectAndFlags /cpython/Objects/memoryobject.c:813:42 
    #4 PyMemoryView_FromObject /cpython/Objects/memoryobject.c:856:12
    #5 memoryview_impl /cpython/Objects/memoryobject.c:1017:12 
    #6 memoryview /cpython/Objects/clinic/memoryobject.c.h:63:20 
    #7 type_call /cpython/Objects/typeobject.c:2472:11 
    #8 _PyObject_MakeTpCall /cpython/Objects/call.c:242:18
    #9 _PyObject_VectorcallTstate /cpython/./Include/internal/pycore_call.h:142:16)
    #10 PyObject_Vectorcall /cpython/Objects/call.c:327:12
    #11 _Py_VectorCall_StackRefSteal /cpython/Python/ceval.c:726:11 
    #12 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:4559:35 

  Previous write of size 8 at 0x7fffb6b50208 by thread T7:
    #0 _ctypes_resize_impl /cpython/./Modules/_ctypes/callproc.c
    #1 _ctypes_resize /cpython/./Modules/_ctypes/clinic/callproc.c.h:139:20
    #2 _Py_BuiltinCallFast_StackRef /cpython/Python/ceval.c:817:11
    #3 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:2510:35
    #4 _PyEval_EvalFrame /cpython/./Include/internal/pycore_ceval.h:122:16

SUMMARY: ThreadSanitizer: data race /cpython/./Modules/_ctypes/_ctypes.c:3129:23 in PyCData_NewGetBuffer
==================
CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs
  • gh-157759

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

从 Modules/_ctypes/_ctypes.c 中的 PyCData_NewGetBuffer 开始,并将其对 b_ptr 的访问与 Modules/_ctypes/callproc.c 中 _ctypes_resize_impl 的临界区进行比较。使用 ThreadSanitizer 运行提供的多线程复现程序。当 memoryview/ctypes.resize 竞态不再被报告,同时现有行为保持不变时,即表示完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
c, python
领域
backend
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
描述清楚
新手友好度
30/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。