python / python/cpython

Segfault in `_Unpickler_MemoPut` (`_pickle.c`) from `Unpickler.memo` assignment with free-threading

未關閉
#150,505 5 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

interpreter-core topic-free-threading type-bug
主要語言
Python
星號
77.2k
分支
35.9k
PR 指標待擷取

描述

Bug report

Bug description:

Unpickler_set_memo modifies self->memo and self->memo_size without any critical section.

When two threads set .memo on the same Unpickler instance concurrently, one can set self->memo = NULL while another is still indexing into it via _Unpickler_MemoPut, leading to a NULL-pointer dereference.

Reproducer:

import io
import pickle
from threading import Thread

shared_unpickler = pickle.Unpickler(io.BytesIO(b''))

def assign():
    for _ in range(10000):
        shared_unpickler.memo = {j: j for j in range(100)}

if __name__ == "__main__":
    threads = [Thread(target=assign) for _ in range(8)]
    for t in threads: t.start()
    for t in threads: t.join()

TSAN report:

WARNING: ThreadSanitizer: data race (pid=214122)
  Read of size 8 at 0x7dca529b9130 by thread T2:
    #0 _Unpickler_MemoPut /home/cpython/./Modules/_pickle.c:1631:22 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x2122f) 
    #1 Unpickler_set_memo /home/cpython/./Modules/_pickle.c:7804:17 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x26b77) 
    #2 getset_set /home/cpython/Objects/descrobject.c:250:16 (python3.16t+0x22d8fb) 
    #3 _PyObject_GenericSetAttrWithDict /home/cpython/Objects/object.c:2049:19 (python3.16t+0x2d844b) 
    #4 PyObject_GenericSetAttr /home/cpython/Objects/object.c:2120:12 (python3.16t+0x2d8d4b) 
    #5 Unpickler_setattr /home/cpython/./Modules/_pickle.c:7857:12 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x24eca) 
    #6 PyObject_SetAttr /home/cpython/Objects/object.c:1533:15 (python3.16t+0x2d480f) 
    #7 _PyEval_EvalFrameDefault /home/cpython/Python/generated_cases.c.h:11536:27 (python3.16t+0x45231a) 
    #8 _PyEval_EvalFrame /home/cpython/./Include/internal/pycore_ceval.h:122:16 (python3.16t+0x436463) 
    #9 _PyEval_Vector /home/cpython/Python/ceval.c:2134:12 (python3.16t+0x436463)
    #10 _PyFunction_Vectorcall /home/cpython/Objects/call.c (python3.16t+0x215013) 
    #11 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x216b11) 
    #12 _PyObject_VectorcallPrepend /home/cpython/Objects/call.c:855:20 (python3.16t+0x216b11)
    #13 method_vectorcall /home/cpython/Objects/classobject.c:55:12 (python3.16t+0x21a132) 
    #14 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x48ec15) 
    #15 context_run /home/cpython/Python/context.c:728:29 (python3.16t+0x48ec15)
    #16 method_vectorcall_FASTCALL_KEYWORDS /home/cpython/Objects/descrobject.c:421:24 (python3.16t+0x22e84c) 
    #17 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x2149a8) 
    #18 PyObject_Vectorcall /home/cpython/Objects/call.c:327:12 (python3.16t+0x2149a8)
    #19 _Py_VectorCallInstrumentation_StackRefSteal /home/cpython/Python/ceval.c:766:11 (python3.16t+0x4370c8) 
    #20 _PyEval_EvalFrameDefault /home/cpython/Python/generated_cases.c.h:1846:35 (python3.16t+0x43d022) 
    #21 _PyEval_EvalFrame /home/cpython/./Include/internal/pycore_ceval.h:122:16 (python3.16t+0x436463) 
    #22 _PyEval_Vector /home/cpython/Python/ceval.c:2134:12 (python3.16t+0x436463)
    #23 _PyFunction_Vectorcall /home/cpython/Objects/call.c (python3.16t+0x215013) 
    #24 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x216b11) 
    #25 _PyObject_VectorcallPrepend /home/cpython/Objects/call.c:855:20 (python3.16t+0x216b11)
    #26 method_vectorcall /home/cpython/Objects/classobject.c:55:12 (python3.16t+0x21a132) 
    #27 _PyVectorcall_Call /home/cpython/Objects/call.c:273:16 (python3.16t+0x214ca2) 
    #28 _PyObject_Call /home/cpython/Objects/call.c:348:16 (python3.16t+0x214ca2)
    #29 PyObject_Call /home/cpython/Objects/call.c:373:12 (python3.16t+0x214d07) 
    #30 thread_run /home/cpython/./Modules/_threadmodule.c:388:21 (python3.16t+0x61a708) 
    #31 pythread_wrapper /home/cpython/Python/thread_pthread.h:234:5 (python3.16t+0x5473bb) 

  Previous write of size 8 at 0x7dca529b9130 by thread T1:
    #0 Unpickler_set_memo /home/cpython/./Modules/_pickle.c:7816:21 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x26cc4) 
    #1 getset_set /home/cpython/Objects/descrobject.c:250:16 (python3.16t+0x22d8fb) 
    #2 _PyObject_GenericSetAttrWithDict /home/cpython/Objects/object.c:2049:19 (python3.16t+0x2d844b) 
    #3 PyObject_GenericSetAttr /home/cpython/Objects/object.c:2120:12 (python3.16t+0x2d8d4b) 
    #4 Unpickler_setattr /home/cpython/./Modules/_pickle.c:7857:12 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x24eca) 
    #5 PyObject_SetAttr /home/cpython/Objects/object.c:1533:15 (python3.16t+0x2d480f) 
    #6 _PyEval_EvalFrameDefault /home/cpython/Python/generated_cases.c.h:11536:27 (python3.16t+0x45231a) 
    #7 _PyEval_EvalFrame /home/cpython/./Include/internal/pycore_ceval.h:122:16 (python3.16t+0x436463) 
    #8 _PyEval_Vector /home/cpython/Python/ceval.c:2134:12 (python3.16t+0x436463)
    #9 _PyFunction_Vectorcall /home/cpython/Objects/call.c (python3.16t+0x215013) 
    #10 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x216b11) 
    #11 _PyObject_VectorcallPrepend /home/cpython/Objects/call.c:855:20 (python3.16t+0x216b11)
    #12 method_vectorcall /home/cpython/Objects/classobject.c:55:12 (python3.16t+0x21a132) 
    #13 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x48ec15) 
    #14 context_run /home/cpython/Python/context.c:728:29 (python3.16t+0x48ec15)
    #15 method_vectorcall_FASTCALL_KEYWORDS /home/cpython/Objects/descrobject.c:421:24 (python3.16t+0x22e84c) 
    #16 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x2149a8) 
    #17 PyObject_Vectorcall /home/cpython/Objects/call.c:327:12 (python3.16t+0x2149a8)
    #18 _Py_VectorCallInstrumentation_StackRefSteal /home/cpython/Python/ceval.c:766:11 (python3.16t+0x4370c8) 
    #19 _PyEval_EvalFrameDefault /home/cpython/Python/generated_cases.c.h:1846:35 (python3.16t+0x43d022) 
    #20 _PyEval_EvalFrame /home/cpython/./Include/internal/pycore_ceval.h:122:16 (python3.16t+0x436463) 
    #21 _PyEval_Vector /home/cpython/Python/ceval.c:2134:12 (python3.16t+0x436463)
    #22 _PyFunction_Vectorcall /home/cpython/Objects/call.c (python3.16t+0x215013) 
    #23 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x216b11) 
    #24 _PyObject_VectorcallPrepend /home/cpython/Objects/call.c:855:20 (python3.16t+0x216b11)
    #25 method_vectorcall /home/cpython/Objects/classobject.c:55:12 (python3.16t+0x21a132) 
    #26 _PyVectorcall_Call /home/cpython/Objects/call.c:273:16 (python3.16t+0x214ca2) 
    #27 _PyObject_Call /home/cpython/Objects/call.c:348:16 (python3.16t+0x214ca2)
    #28 PyObject_Call /home/cpython/Objects/call.c:373:12 (python3.16t+0x214d07) 
    #29 thread_run /home/cpython/./Modules/_threadmodule.c:388:21 (python3.16t+0x61a708) 
    #30 pythread_wrapper /home/cpython/Python/thread_pthread.h:234:5 (python3.16t+0x5473bb) 

SUMMARY: ThreadSanitizer: data race /home/cpython/./Modules/_pickle.c:1631:22 in _Unpickler_MemoPut
==================

==214455==ERROR: ThreadSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7afb5f96f25a bp 0x7afb187fd6f0 sp 0x7afb187fd690 T214460)
==214455==The signal is caused by a READ memory access.
==214455==Hint: address points to the zero page.
    #0 _Unpickler_MemoPut /home/cpython/./Modules/_pickle.c:1657:16 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x2125a) 
    #1 Unpickler_set_memo /home/cpython/./Modules/_pickle.c:7804:17 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x26b77) 
    #2 getset_set /home/cpython/Objects/descrobject.c:250:16 (python3.16t+0x22d8fb) 
    #3 _PyObject_GenericSetAttrWithDict /home/cpython/Objects/object.c:2049:19 (python3.16t+0x2d844b) 
    #4 PyObject_GenericSetAttr /home/cpython/Objects/object.c:2120:12 (python3.16t+0x2d8d4b) 
    #5 Unpickler_setattr /home/cpython/./Modules/_pickle.c:7857:12 (_pickle.cpython-316t-x86_64-linux-gnu.so+0x24eca) 
    #6 PyObject_SetAttr /home/cpython/Objects/object.c:1533:15 (python3.16t+0x2d480f) 
    #7 _PyEval_EvalFrameDefault /home/cpython/Python/generated_cases.c.h:11536:27 (python3.16t+0x45231a) 
    #8 _PyEval_EvalFrame /home/cpython/./Include/internal/pycore_ceval.h:122:16 (python3.16t+0x436463) 
    #9 _PyEval_Vector /home/cpython/Python/ceval.c:2134:12 (python3.16t+0x436463)
    #10 _PyFunction_Vectorcall /home/cpython/Objects/call.c (python3.16t+0x215013) 
    #11 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x216b11) 
    #12 _PyObject_VectorcallPrepend /home/cpython/Objects/call.c:855:20 (python3.16t+0x216b11)
    #13 method_vectorcall /home/cpython/Objects/classobject.c:55:12 (python3.16t+0x21a132) 
    #14 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x48ec15) 
    #15 context_run /home/cpython/Python/context.c:728:29 (python3.16t+0x48ec15)
    #16 method_vectorcall_FASTCALL_KEYWORDS /home/cpython/Objects/descrobject.c:421:24 (python3.16t+0x22e84c) 
    #17 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x2149a8) 
    #18 PyObject_Vectorcall /home/cpython/Objects/call.c:327:12 (python3.16t+0x2149a8)
    #19 _Py_VectorCallInstrumentation_StackRefSteal /home/cpython/Python/ceval.c:766:11 (python3.16t+0x4370c8) 
    #20 _PyEval_EvalFrameDefault /home/cpython/Python/generated_cases.c.h:1846:35 (python3.16t+0x43d022) 
    #21 _PyEval_EvalFrame /home/cpython/./Include/internal/pycore_ceval.h:122:16 (python3.16t+0x436463) 
    #22 _PyEval_Vector /home/cpython/Python/ceval.c:2134:12 (python3.16t+0x436463)
    #23 _PyFunction_Vectorcall /home/cpython/Objects/call.c (python3.16t+0x215013) 
    #24 _PyObject_VectorcallTstate /home/cpython/./Include/internal/pycore_call.h:144:11 (python3.16t+0x216b11) 
    #25 _PyObject_VectorcallPrepend /home/cpython/Objects/call.c:855:20 (python3.16t+0x216b11)
    #26 method_vectorcall /home/cpython/Objects/classobject.c:55:12 (python3.16t+0x21a132) 
    #27 _PyVectorcall_Call /home/cpython/Objects/call.c:273:16 (python3.16t+0x214ca2) 
    #28 _PyObject_Call /home/cpython/Objects/call.c:348:16 (python3.16t+0x214ca2)
    #29 PyObject_Call /home/cpython/Objects/call.c:373:12 (python3.16t+0x214d07) 
    #30 thread_run /home/cpython/./Modules/_threadmodule.c:388:21 (python3.16t+0x61a708) 
    #31 pythread_wrapper /home/cpython/Python/thread_pthread.h:234:5 (python3.16t+0x5473bb) 
    #32 __tsan_thread_start_func <null> (python3.16t+0xf9182) 
    #33 start_thread nptl/pthread_create.c:447:8 (libc.so.6+0x9caa3) (BuildId: 8e9fd827446c24067541ac5390e6f527fb5947bb)
    #34 clone3 misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:78 (libc.so.6+0x129c6b) (BuildId: 8e9fd827446c24067541ac5390e6f527fb5947bb)

ThreadSanitizer can not provide additional info.
SUMMARY: ThreadSanitizer: SEGV /home/cpython/./Modules/_pickle.c:1657:16 in _Unpickler_MemoPut
==214455==ABORTING
CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs
  • gh-150550

貢獻指南

開啟貢獻指南

從這裡開始

  1. 先讀完整個 Issue，再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫，在一個分支上完成修改。
  4. 送出 Pull Request，並在描述裡引用這個 Issue 編號。

研究方向

從 Modules/_pickle.c 中的 _Unpickler_MemoPut 和 Unpickler_set_memo 開始，使用提供的執行緒 reproducer 和 TSAN 報告來追蹤競爭條件。檢查連結的 PR gh-150550，並驗證並行的 Unpickler.memo 指派不再產生 data race 或 NULL 指標解參照。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
c, python
領域
backend
Issue 類型
缺陷
難度
4/5
預估耗時
3-5 天
活躍度
停滯
描述清晰度
基本清楚
新手友好度
25/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。