python / python/cpython

Concurrent calls to `Counter.update` can cause a use after free

Open
#151,633 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

extension-modules topic-free-threading type-crash
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR metrics pending

Description

Crash report

What happened?

In Counter.update, _collections__count_elements_impl (called through _count_elements then _collections__count_elements) has a fast path for plain dicts that uses _PyDict_GetItem_KnownHash to obtain a borrowed reference to the current value. The current value is then incremented using PyNumber_Add (but after the dict lock is released).

https://github.com/python/cpython/blob/d986124d83465190987357f987ee24bd7a817cac/Modules/_collectionsmodule.c#L2598-L2614

When multiple threads call update on the same Counter concurrently, this can cause a user after free.

  • _PyDict_GetItem_KnownHash acquires the per-dict lock, looks up oldval, then releases the lock before returning the borrowed reference. Between that return and the next Py_INCREF(oldval), a second thread can:
    • call _PyDict_GetItem_KnownHash on the same key and get the same oldval
    • do Py_INCREF(oldval) / PyNumber_Add(oldval, one) / _PyDict_SetItem_KnownHash
    • then cause the dict to decref and free oldval (if the dict held the last reference).
  • The first thread then executes Py_INCREF(oldval) and PyNumber_Add(oldval, one) on an oldval that has been freed.

The following reproducer run under TSan shows many reports, but simply running it on a free-threaded build produces a hard crash.

import threading
from collections import Counter

def do() -> None:
    c: Counter[str] = Counter()
    data = ["x"] * 5000

    def worker() -> None:
        c.update(data)

    threads = [threading.Thread(target=worker) for _ in range(4)]
    for t in threads: t.start()
    for t in threads: t.join()

for _ in range(50):
    do()

Under free-threaded Python

$ ./python.exe ./new_bug_counter.py 
zsh: segmentation fault  ../cpython-nogil/python.exe ./new_bug_counter.py

Under TSAN (CPython main, free-threaded build):

==================
WARNING: ThreadSanitizer: data race (pid=69202)
  Read of size 8 at 0x0001120401e0 by thread T4:
    #0 _PyFreeList_PopNoStats pycore_freelist.h:80 (python.exe:arm64+0x1000f2318)
    #1 _PyFreeList_Pop pycore_freelist.h:89 (python.exe:arm64+0x1000f2318)
    #2 _PyLong_FromMedium longobject.c:256 (python.exe:arm64+0x1000f2318)
    #3 _PyLong_FromSTwoDigits longobject.c:320 (python.exe:arm64+0x1000ffb7c)
    #4 long_add longobject.c:3838 (python.exe:arm64+0x1000ffb7c)
    #5 long_add_method longobject.c:3878 (python.exe:arm64+0x10010277c)
    #6 binary_op1 abstract.c:974 (python.exe:arm64+0x10005edb4)
    #7 PyNumber_Add abstract.c:1140 (python.exe:arm64+0x10005eb10)
    #8 _collections__count_elements_impl _collectionsmodule.c:2602 (python.exe:arm64+0x1003f51c8)
    #9 _collections__count_elements _collectionsmodule.c.h:593 (python.exe:arm64+0x1003f51c8)
    #10 cfunction_vectorcall_FASTCALL methodobject.c:449 (python.exe:arm64+0x1001379dc)
    #11 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100090e80)
    #12 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090e80)
    #13 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
    #14 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
    #15 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #16 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #17 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #18 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #19 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #20 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #21 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
    #22 context_run context.c:728 (python.exe:arm64+0x1002dce38)
    #23 method_vectorcall_FASTCALL_KEYWORDS descrobject.c:421 (python.exe:arm64+0x1000a8644)
    #24 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100090e80)
    #25 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090e80)
    #26 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
    #27 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
    #28 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #29 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #30 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #31 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #32 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #33 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #34 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
    #35 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
    #36 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
    #37 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
    #38 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)

  Previous atomic write of size 8 at 0x0001120401e0 by thread T2:
    #0 _Py_atomic_store_ptr_relaxed pyatomic_gcc.h:509 (python.exe:arm64+0x1000ffcb0)
    #1 _PyFreeList_Push pycore_freelist.h:56 (python.exe:arm64+0x1000ffcb0)
    #2 _PyFreeList_Free pycore_freelist.h:69 (python.exe:arm64+0x1000ffcb0)
    #3 long_dealloc longobject.c:3664 (python.exe:arm64+0x1000ffcb0)
    #4 _Py_Dealloc object.c:3312 (python.exe:arm64+0x10013fd5c)
    #5 _Py_DecRefSharedDebug object.c:426 (python.exe:arm64+0x10013ff34)
    #6 _Py_DecRefShared object.c:433 (python.exe:arm64+0x10013ff34)
    #7 Py_DECREF refcount.h:385 (python.exe:arm64+0x1003f5228)
    #8 _collections__count_elements_impl _collectionsmodule.c:2603 (python.exe:arm64+0x1003f5228)
    #9 _collections__count_elements _collectionsmodule.c.h:593 (python.exe:arm64+0x1003f5228)
    #10 cfunction_vectorcall_FASTCALL methodobject.c:449 (python.exe:arm64+0x1001379dc)
    #11 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100090e80)
    #12 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090e80)
    #13 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
    #14 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
    #15 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #16 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #17 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #18 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #19 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #20 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #21 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
    #22 context_run context.c:728 (python.exe:arm64+0x1002dce38)
    #23 method_vectorcall_FASTCALL_KEYWORDS descrobject.c:421 (python.exe:arm64+0x1000a8644)
    #24 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100090e80)
    #25 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090e80)
    #26 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
    #27 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
    #28 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #29 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #30 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #31 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #32 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #33 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #34 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
    #35 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
    #36 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
    #37 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
    #38 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)

  Thread T4 (tid=18691961, running) created by main thread at:
    #0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84)
    #1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x100389300)
    #2 PyThread_start_joinable_thread thread_pthread.h:323 (python.exe:arm64+0x100389138)
    #3 ThreadHandle_start _threadmodule.c:475 (python.exe:arm64+0x10044dd10)
    #4 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10044d7dc)
    #5 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x10044c844)
    #6 cfunction_call methodobject.c:564 (python.exe:arm64+0x100138744)
    #7 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
    #8 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
    #9 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
    #10 _Py_VectorCall_StackRefSteal ceval.c:724 (python.exe:arm64+0x100290228)
    #11 _PyEval_EvalFrameDefault generated_cases.c.h:3528 (python.exe:arm64+0x100299d1c)
    #12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fa34)
    #13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fa34)
    #14 PyEval_EvalCode ceval.c:677 (python.exe:arm64+0x10028fa34)
    #15 run_eval_code_obj pythonrun.c:1369 (python.exe:arm64+0x100366a70)
    #16 run_mod pythonrun.c:1472 (python.exe:arm64+0x1003667cc)
    #17 pyrun_file pythonrun.c:1296 (python.exe:arm64+0x100361d38)
    #18 _PyRun_SimpleFileObject pythonrun.c:518 (python.exe:arm64+0x100361d38)
    #19 _PyRun_AnyFileObject pythonrun.c:81 (python.exe:arm64+0x1003614c8)
    #20 pymain_run_file_obj main.c:411 (python.exe:arm64+0x1003a3328)
    #21 pymain_run_file main.c:430 (python.exe:arm64+0x1003a3328)
    #22 pymain_run_python main.c:715 (python.exe:arm64+0x1003a2658)
    #23 Py_RunMain main.c:796 (python.exe:arm64+0x1003a2658)
    #24 pymain_main main.c:826 (python.exe:arm64+0x1003a2bd0)
    #25 Py_BytesMain main.c:850 (python.exe:arm64+0x1003a2cd0)
    #26 main python.c:15 (python.exe:arm64+0x100000a78)

  Thread T2 (tid=18691959, running) created by main thread at:
    #0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84)
    #1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x100389300)
    #2 PyThread_start_joinable_thread thread_pthread.h:323 (python.exe:arm64+0x100389138)
    #3 ThreadHandle_start _threadmodule.c:475 (python.exe:arm64+0x10044dd10)
    #4 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10044d7dc)
    #5 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x10044c844)
    #6 cfunction_call methodobject.c:564 (python.exe:arm64+0x100138744)
    #7 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
    #8 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
    #9 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
    #10 _Py_VectorCall_StackRefSteal ceval.c:724 (python.exe:arm64+0x100290228)
    #11 _PyEval_EvalFrameDefault generated_cases.c.h:3528 (python.exe:arm64+0x100299d1c)
    #12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fa34)
    #13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fa34)
    #14 PyEval_EvalCode ceval.c:677 (python.exe:arm64+0x10028fa34)
    #15 run_eval_code_obj pythonrun.c:1369 (python.exe:arm64+0x100366a70)
    #16 run_mod pythonrun.c:1472 (python.exe:arm64+0x1003667cc)
    #17 pyrun_file pythonrun.c:1296 (python.exe:arm64+0x100361d38)
    #18 _PyRun_SimpleFileObject pythonrun.c:518 (python.exe:arm64+0x100361d38)
    #19 _PyRun_AnyFileObject pythonrun.c:81 (python.exe:arm64+0x1003614c8)
    #20 pymain_run_file_obj main.c:411 (python.exe:arm64+0x1003a3328)
    #21 pymain_run_file main.c:430 (python.exe:arm64+0x1003a3328)
    #22 pymain_run_python main.c:715 (python.exe:arm64+0x1003a2658)
    #23 Py_RunMain main.c:796 (python.exe:arm64+0x1003a2658)
    #24 pymain_main main.c:826 (python.exe:arm64+0x1003a2bd0)
    #25 Py_BytesMain main.c:850 (python.exe:arm64+0x1003a2cd0)
    #26 main python.c:15 (python.exe:arm64+0x100000a78)

SUMMARY: ThreadSanitizer: data race pycore_freelist.h:80 in _PyFreeList_PopNoStats
==================
==================
WARNING: ThreadSanitizer: data race (pid=69202)
  Read of size 8 at 0x000116040540 by thread T8:
    #0 _PyFreeList_PopNoStats pycore_freelist.h:80 (python.exe:arm64+0x1000f2318)
    #1 _PyFreeList_Pop pycore_freelist.h:89 (python.exe:arm64+0x1000f2318)
    #2 _PyLong_FromMedium longobject.c:256 (python.exe:arm64+0x1000f2318)
    #3 _PyLong_FromSTwoDigits longobject.c:320 (python.exe:arm64+0x1000ffb7c)
    #4 long_add longobject.c:3838 (python.exe:arm64+0x1000ffb7c)
    #5 long_add_method longobject.c:3878 (python.exe:arm64+0x10010277c)
    #6 binary_op1 abstract.c:974 (python.exe:arm64+0x10005edb4)
    #7 PyNumber_Add abstract.c:1140 (python.exe:arm64+0x10005eb10)
    #8 _collections__count_elements_impl _collectionsmodule.c:2602 (python.exe:arm64+0x1003f51c8)
    #9 _collections__count_elements _collectionsmodule.c.h:593 (python.exe:arm64+0x1003f51c8)
    #10 _Py_BuiltinCallFast_StackRef ceval.c:815 (python.exe:arm64+0x1002976c0)
    #11 _PyEval_EvalFrameDefault generated_cases.c.h:2420 (python.exe:arm64+0x1002976c0)
    #12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #14 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #15 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #16 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #17 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #18 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
    #19 context_run context.c:728 (python.exe:arm64+0x1002dce38)
    #20 _PyCallMethodDescriptorFastWithKeywords_StackRef ceval.c:881 (python.exe:arm64+0x10029ac0c)
    #21 _PyEval_EvalFrameDefault generated_cases.c.h:4022 (python.exe:arm64+0x10029ac0c)
    #22 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #23 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #24 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #25 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #26 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #27 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #28 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
    #29 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
    #30 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
    #31 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
    #32 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)

  Previous atomic write of size 8 at 0x000116040540 by thread T7:
    #0 _Py_atomic_store_ptr_relaxed pyatomic_gcc.h:509 (python.exe:arm64+0x1000ffcb0)
    #1 _PyFreeList_Push pycore_freelist.h:56 (python.exe:arm64+0x1000ffcb0)
    #2 _PyFreeList_Free pycore_freelist.h:69 (python.exe:arm64+0x1000ffcb0)
    #3 long_dealloc longobject.c:3664 (python.exe:arm64+0x1000ffcb0)
    #4 _Py_Dealloc object.c:3312 (python.exe:arm64+0x10013fd5c)
    #5 _Py_DecRefSharedDebug object.c:426 (python.exe:arm64+0x10013ff34)
    #6 _Py_DecRefShared object.c:433 (python.exe:arm64+0x10013ff34)
    #7 Py_DECREF refcount.h:385 (python.exe:arm64+0x1003f5228)
    #8 _collections__count_elements_impl _collectionsmodule.c:2603 (python.exe:arm64+0x1003f5228)
    #9 _collections__count_elements _collectionsmodule.c.h:593 (python.exe:arm64+0x1003f5228)
    #10 _Py_BuiltinCallFast_StackRef ceval.c:815 (python.exe:arm64+0x1002976c0)
    #11 _PyEval_EvalFrameDefault generated_cases.c.h:2420 (python.exe:arm64+0x1002976c0)
    #12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #14 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #15 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #16 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #17 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #18 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
    #19 context_run context.c:728 (python.exe:arm64+0x1002dce38)
    #20 _PyCallMethodDescriptorFastWithKeywords_StackRef ceval.c:881 (python.exe:arm64+0x10029ac0c)
    #21 _PyEval_EvalFrameDefault generated_cases.c.h:4022 (python.exe:arm64+0x10029ac0c)
    #22 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #23 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #24 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #25 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #26 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #27 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #28 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
    #29 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
    #30 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
    #31 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
    #32 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)

  Thread T8 (tid=18691990, running) created by main thread at:
    #0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84)
    #1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x100389300)
    #2 PyThread_start_joinable_thread thread_pthread.h:323 (python.exe:arm64+0x100389138)
    #3 ThreadHandle_start _threadmodule.c:475 (python.exe:arm64+0x10044dd10)
    #4 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10044d7dc)
    #5 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x10044c844)
    #6 cfunction_call methodobject.c:564 (python.exe:arm64+0x100138744)
    #7 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
    #8 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
    #9 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
    #10 _Py_VectorCall_StackRefSteal ceval.c:724 (python.exe:arm64+0x100290228)
    #11 _PyEval_EvalFrameDefault generated_cases.c.h:3528 (python.exe:arm64+0x100299d1c)
    #12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fa34)
    #13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fa34)
    #14 PyEval_EvalCode ceval.c:677 (python.exe:arm64+0x10028fa34)
    #15 run_eval_code_obj pythonrun.c:1369 (python.exe:arm64+0x100366a70)
    #16 run_mod pythonrun.c:1472 (python.exe:arm64+0x1003667cc)
    #17 pyrun_file pythonrun.c:1296 (python.exe:arm64+0x100361d38)
    #18 _PyRun_SimpleFileObject pythonrun.c:518 (python.exe:arm64+0x100361d38)
    #19 _PyRun_AnyFileObject pythonrun.c:81 (python.exe:arm64+0x1003614c8)
    #20 pymain_run_file_obj main.c:411 (python.exe:arm64+0x1003a3328)
    #21 pymain_run_file main.c:430 (python.exe:arm64+0x1003a3328)
    #22 pymain_run_python main.c:715 (python.exe:arm64+0x1003a2658)
    #23 Py_RunMain main.c:796 (python.exe:arm64+0x1003a2658)
    #24 pymain_main main.c:826 (python.exe:arm64+0x1003a2bd0)
    #25 Py_BytesMain main.c:850 (python.exe:arm64+0x1003a2cd0)
    #26 main python.c:15 (python.exe:arm64+0x100000a78)

  Thread T7 (tid=18691989, running) created by main thread at:
    #0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84)
    #1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x100389300)
    #2 PyThread_start_joinable_thread thread_pthread.h:323 (python.exe:arm64+0x100389138)
    #3 ThreadHandle_start _threadmodule.c:475 (python.exe:arm64+0x10044dd10)
    #4 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10044d7dc)
    #5 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x10044c844)
    #6 cfunction_call methodobject.c:564 (python.exe:arm64+0x100138744)
    #7 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
    #8 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
    #9 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
    #10 _Py_VectorCall_StackRefSteal ceval.c:724 (python.exe:arm64+0x100290228)
    #11 _PyEval_EvalFrameDefault generated_cases.c.h:3528 (python.exe:arm64+0x100299d1c)
    #12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fa34)
    #13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fa34)
    #14 PyEval_EvalCode ceval.c:677 (python.exe:arm64+0x10028fa34)
    #15 run_eval_code_obj pythonrun.c:1369 (python.exe:arm64+0x100366a70)
    #16 run_mod pythonrun.c:1472 (python.exe:arm64+0x1003667cc)
    #17 pyrun_file pythonrun.c:1296 (python.exe:arm64+0x100361d38)
    #18 _PyRun_SimpleFileObject pythonrun.c:518 (python.exe:arm64+0x100361d38)
    #19 _PyRun_AnyFileObject pythonrun.c:81 (python.exe:arm64+0x1003614c8)
    #20 pymain_run_file_obj main.c:411 (python.exe:arm64+0x1003a3328)
    #21 pymain_run_file main.c:430 (python.exe:arm64+0x1003a3328)
    #22 pymain_run_python main.c:715 (python.exe:arm64+0x1003a2658)
    #23 Py_RunMain main.c:796 (python.exe:arm64+0x1003a2658)
    #24 pymain_main main.c:826 (python.exe:arm64+0x1003a2bd0)
    #25 Py_BytesMain main.c:850 (python.exe:arm64+0x1003a2cd0)
    #26 main python.c:15 (python.exe:arm64+0x100000a78)

SUMMARY: ThreadSanitizer: data race pycore_freelist.h:80 in _PyFreeList_PopNoStats
==================
ThreadSanitizer:DEADLYSIGNAL
==================
WARNING: ThreadSanitizer: data race (pid=69202)
  Write of size 8 at 0x00011e04af40 by thread T5:
    #0 _PyLong_SetSignAndDigitCount pycore_long.h:307 (python.exe:arm64+0x1000f242c)
    #1 _PyLong_FromMedium longobject.c:267 (python.exe:arm64+0x1000f242c)
    #2 _PyLong_FromSTwoDigits longobject.c:320 (python.exe:arm64+0x1000ffb7c)
    #3 long_add longobject.c:3838 (python.exe:arm64+0x1000ffb7c)
    #4 long_add_method longobject.c:3878 (python.exe:arm64+0x10010277c)
    #5 binary_op1 abstract.c:974 (python.exe:arm64+0x10005edb4)
    #6 PyNumber_Add abstract.c:1140 (python.exe:arm64+0x10005eb10)
    #7 _collections__count_elements_impl _collectionsmodule.c:2602 (python.exe:arm64+0x1003f51c8)
    #8 _collections__count_elements _collectionsmodule.c.h:593 (python.exe:arm64+0x1003f51c8)
    #9 _Py_BuiltinCallFast_StackRef ceval.c:815 (python.exe:arm64+0x1002976c0)
    #10 _PyEval_EvalFrameDefault generated_cases.c.h:2420 (python.exe:arm64+0x1002976c0)
    #11 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #12 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #13 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #14 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #15 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #16 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #17 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
    #18 context_run context.c:728 (python.exe:arm64+0x1002dce38)
    #19 _PyCallMethodDescriptorFastWithKeywords_StackRef ceval.c:881 (python.exe:arm64+0x10029ac0c)
    #20 _PyEval_EvalFrameDefault generated_cases.c.h:4022 (python.exe:arm64+0x10029ac0c)
    #21 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #22 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #23 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #24 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #25 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #26 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #27 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
    #28 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
    #29 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
    #30 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
    #31 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)

  Previous write of size 8 at 0x00011e04af40 by thread T6:
    #0 _PyLong_SetSignAndDigitCount pycore_long.h:307 (python.exe:arm64+0x1000f242c)
    #1 _PyLong_FromMedium longobject.c:267 (python.exe:arm64+0x1000f242c)
    #2 _PyLong_FromSTwoDigits longobject.c:320 (python.exe:arm64+0x1000ffb7c)
    #3 long_add longobject.c:3838 (python.exe:arm64+0x1000ffb7c)
    #4 long_add_method longobject.c:3878 (python.exe:arm64+0x10010277c)
    #5 binary_op1 abstract.c:974 (python.exe:arm64+0x10005edb4)
    #6 PyNumber_Add abstract.c:1140 (python.exe:arm64+0x10005eb10)
    #7 _collections__count_elements_impl _collectionsmodule.c:2602 (python.exe:arm64+0x1003f51c8)
    #8 _collections__count_elements _collectionsmodule.c.h:593 (python.exe:arm64+0x1003f51c8)
    #9 _Py_BuiltinCallFast_StackRef ceval.c:815 (python.exe:arm64+0x1002976c0)
    #10 _PyEval_EvalFrameDefault generated_cases.c.h:4022 (python.exe:arm64+0x1002976c0)
    #11 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #12 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #13 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #14 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #15 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #16 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #17 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
    #18 context_run context.c:728 (python.exe:arm64+0x1002dce38)
    #19 _PyCallMethodDescriptorFastWithKeywords_StackRef ceval.c:881 (python.exe:arm64+0x10029ac0c)
    #20 _PyEval_EvalFrameDefault generated_cases.c.h:4022 (python.exe:arm64+0x10029ac0c)
    #21 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #22 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #23 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #24 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #25 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #26 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #27 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
    #28 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
    #29 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
    #30 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
    #31 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)

  Thread T5 (tid=18691987, running) created by main thread at:
    #0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84)
    #1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x100389300)
    #2 PyThread_start_joinable_thread thread_pthread.h:323 (python.exe:arm64+0x100389138)
    #3 ThreadHandle_start _threadmodule.c:475 (python.exe:arm64+0x10044dd10)
    #4 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10044d7dc)
    #5 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x10044c844)
    #6 cfunction_call methodobject.c:564 (python.exe:arm64+0x100138744)
    #7 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
    #8 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
    #9 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
    #10 _Py_VectorCall_StackRefSteal ceval.c:724 (python.exe:arm64+0x100290228)
    #11 _PyEval_EvalFrameDefault generated_cases.c.h:3528 (python.exe:arm64+0x100299d1c)
    #12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fa34)
    #13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fa34)
    #14 PyEval_EvalCode ceval.c:677 (python.exe:arm64+0x10028fa34)
    #15 run_eval_code_obj pythonrun.c:1369 (python.exe:arm64+0x100366a70)
    #16 run_mod pythonrun.c:1472 (python.exe:arm64+0x1003667cc)
    #17 pyrun_file pythonrun.c:1296 (python.exe:arm64+0x100361d38)
    #18 _PyRun_SimpleFileObject pythonrun.c:518 (python.exe:arm64+0x100361d38)
    #19 _PyRun_AnyFileObject pythonrun.c:81 (python.exe:arm64+0x1003614c8)
    #20 pymain_run_file_obj main.c:411 (python.exe:arm64+0x1003a3328)
    #21 pymain_run_file main.c:430 (python.exe:arm64+0x1003a3328)
    #22 pymain_run_python main.c:715 (python.exe:arm64+0x1003a2658)
    #23 Py_RunMain main.c:796 (python.exe:arm64+0x1003a2658)
    #24 pymain_main main.c:826 (python.exe:arm64+0x1003a2bd0)
    #25 Py_BytesMain main.c:850 (python.exe:arm64+0x1003a2cd0)
    #26 main python.c:15 (python.exe:arm64+0x100000a78)

  Thread T6 (tid=18691988, running) created by main thread at:
    #0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84)
    #1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x100389300)
    #2 PyThread_start_joinable_thread thread_pthread.h:323 (python.exe:arm64+0x100389138)
    #3 ThreadHandle_start _threadmodule.c:475 (python.exe:arm64+0x10044dd10)
    #4 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10044d7dc)
    #5 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x10044c844)
    #6 cfunction_call methodobject.c:564 (python.exe:arm64+0x100138744)
    #7 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
    #8 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
    #9 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
    #10 _Py_VectorCall_StackRefSteal ceval.c:724 (python.exe:arm64+0x100290228)
    #11 _PyEval_EvalFrameDefault generated_cases.c.h:3528 (python.exe:arm64+0x100299d1c)
    #12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fa34)
    #13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fa34)
    #14 PyEval_EvalCode ceval.c:677 (python.exe:arm64+0x10028fa34)
    #15 run_eval_code_obj pythonrun.c:1369 (python.exe:arm64+0x100366a70)
    #16 run_mod pythonrun.c:1472 (python.exe:arm64+0x1003667cc)
    #17 pyrun_file pythonrun.c:1296 (python.exe:arm64+0x100361d38)
    #18 _PyRun_SimpleFileObject pythonrun.c:518 (python.exe:arm64+0x100361d38)
    #19 _PyRun_AnyFileObject pythonrun.c:81 (python.exe:arm64+0x1003614c8)
    #20 pymain_run_file_obj main.c:411 (python.exe:arm64+0x1003a3328)
    #21 pymain_run_file main.c:430 (python.exe:arm64+0x1003a3328)
    #22 pymain_run_python main.c:715 (python.exe:arm64+0x1003a2658)
    #23 Py_RunMain main.c:796 (python.exe:arm64+0x1003a2658)
    #24 pymain_main main.c:826 (python.exe:arm64+0x1003a2bd0)
    #25 Py_BytesMain main.c:850 (python.exe:arm64+0x1003a2cd0)
    #26 main python.c:15 (python.exe:arm64+0x100000a78)

SUMMARY: ThreadSanitizer: data race pycore_long.h:307 in _PyLong_SetSignAndDigitCount
==================
ThreadSanitizer:DEADLYSIGNAL
ThreadSanitizer: nested bug in the same thread, aborting.
CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Output from running 'python -VV' on the command line:

No response

Linked PRs
  • gh-151634

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in Modules/_collectionsmodule.c at _collections__count_elements_impl, especially the plain-dict fast path around lines 2598-2614. Reproduce the failure with the provided Counter.update threading script on a free-threaded build, then verify that concurrent updates no longer trigger the reported use-after-free or crash.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.