Free-threading data races in CPython: 15 findings (9 new + 6 residual of existing FT work)
Chưa có ai nhận issue này.
- Ngôn ngữ chính
- Python
- Star
- 77.2k
- Fork
- 35.9k
- Chỉ số merge pull request
- Chỉ số pull request đang chờ
Mô tả
Crash report
What happened?
ThreadSanitizer fuzzing of free-threaded CPython main turned up 15 data races where a built-in object or interpreter-global state is accessed concurrently without the atomics or critical section the surrounding code (or the documentation) already assumes. Each has a minimal, stdlib-only reproducer, the raw TSan report, a root-cause analysis against current-main source, and a suggested fix, published as a self-contained gist (linked below).
They split into two groups: 9 are new defects, and 6 are residuals of free-threading work that is already merged or documented — a specific reader/field/path that a completed conversion left behind. The residuals are the lowest-risk to fix (the pattern and the fix are already in the tree next to them); the related issue/PR is named in the table.
I'm filing them under one umbrella so they can be picked off individually rather than flooding the tracker. To take one: open a normal CPython issue or PR and drop a comment here with the link — I'll mark it in the table. If any is a duplicate or a non-bug, say so and I'll annotate it.
Found with fusil's --tsan mode (fusil originally by @vstinner). Reports and reproducers were drafted with AI assistance (Claude Code) and then reviewed and re-verified by hand — see Disclosure.
Reproducing
- Found on
main(3.16.0a0) on a--disable-gil --with-thread-sanitizerdebug build. These are concurrency bugs, not tied to an exact revision. - Each gist ships a minimal stdlib-only
TSAN-NNNN-repro.py. Run it on a free-threaded TSan build withPYTHON_GIL=0; a real race exits non-zero and printsWARNING: ThreadSanitizer: data race. (TSan needs ASLR reduced — e.g.setarch -R— and an unlimitedRLIMIT_AS, or it runs degraded.) - Most are value-benign on aligned hardware but are genuine C-level data races (formally UB, and TSan-reported); a few carry a latent UAF/leak/crash, called out per report. All 15 reproduce in isolation with a stdlib-only script (some probabilistically — loop them; the debug-only finalization assert TSAN-0034 hits ~44 %/run).
New free-threading defects (9)
Status legend: blank = not yet filed · #N = open issue/PR · #N FIXED = filed and fixed.
| Report | Race | Suggested fix | Status |
|---|---|---|---|
| TSAN-0001 | multibytecodec.c: MultibyteIncrementalDecoder.getstate()/reset()/decode() on a shared decoder race its pending/pendingsize/state fields — the incremental codecs have no critical sections (incl. the iso-2022/HZ face) |
per-object critical sections on the incremental codec methods | |
| TSAN-0005 | _decimal.c: Decimal.__hash__ writes its lazy hash cache (self->hash) with plain stores; concurrent hash() of a shared Decimal races |
relaxed atomics on self->hash |
#154037 (PR #154045) |
| TSAN-0006 | itertoolsmodule.c: count.__repr__ plain-reads cnt while count_next writes it with an atomic CAS — the writer was hardened, the reader missed |
_Py_atomic_load_ssize_relaxed in count_repr |
#153908 FIXED |
| TSAN-0011 | sysmodule.c: sys.addaudithook lazily creates interp->audit_hooks with no lock, racing should_audit on every audit event — security-relevant (PEP 578); the sibling C-level hook list is already mutex-guarded |
serialize under runtime->audit_hooks.mutex; atomics on the pointer |
#154431 FIXED |
| TSAN-0018 | dictobject.c: readers of a shared dict's dk_nentries use plain loads while setattr/insert bump it atomically — including the public PyDict_Next C-API (via _PyType_GetSubclasses), so any extension iterating a shared dict is exposed. LOAD_KEYS_NENTRIES already exists at :237 |
LOAD_KEYS_NENTRIES at the reader sites (_PyObject_IsInstanceDictEmpty, clear_lock_held, _PyDict_Next) |
#153881 FIXED |
| TSAN-0030 | instrumentation.c: sys.monitoring.use_tool_id() is an unsynchronized check-then-act on the interpreter-global monitoring_tool_names[] — both threads pass the guard → leak + dup ownership; free_tool_id's Py_CLEAR is a UAF/double-free. All four accessors are unlocked |
lock/atomic the tool-id registry; fix all four accessors together | #154348 (PR #154459) |
| TSAN-0031 | _elementtree.c: concurrent feed of one shared TreeBuilder races its parse state (this/last/data/index/stack) — the module has zero critical sections, yet declares Py_MOD_GIL_NOT_USED |
@critical_section the whole TreeBuilder feed path (start/data/end/comment/pi/close) |
(abandoned PR gh-145569 covered only handle_end) |
| TSAN-0035 | socketmodule.c: sock_timeout is read/written with plain accesses (gettimeout() vs setblocking()) — the one per-socket scalar the module's free-threading conversion (gh-128277) missed, while the sibling sock_fd and state->defaulttimeout were made atomic. _ssl.c inherits it |
get/set_sock_timeout over _Py_atomic_{load,store}_int64_relaxed, mirroring sock_fd |
#153935 (PR #153940) |
| TSAN-0036 | instrumentation.c/ceval.c: the eval loop reads a code object's active_monitors.tools[] lock-free (no_tools_for_local_event, via gen_close) while _Py_Instrument replaces the struct under LOCK_CODE — which the eval loop never takes; lazy re-instrumentation runs with the world running |
relaxed atomics on the tools[] bytes (matching the file's opcode discipline) |
(PR gh-136994 did exactly this for the bytecode tool bytes but not active_monitors) |
Residuals of existing / documented free-threading work (6)
These are a reader/field/path left behind by a completed (or documented) conversion — likely best handled as a follow-up to the named issue rather than as fresh bugs.
| Report | Race | Related upstream | Status |
|---|---|---|---|
| TSAN-0002 | _zstd: ZstdCompressor.last_mode is stored plain but read lock-free via its Py_T_INT member descriptor |
residual of gh-133885 / gh-134289 (added locks, left last_mode plain) |
#154407 |
| TSAN-0013 | shared list: non-atomic readers (Py_SIZE/unpack, stringlib_bytes_join, marshal) race list_resize's atomic ob_item/ob_size publish |
reader-side residual of gh-129069; contract documented in gh-142519 | #154516 |
| TSAN-0014 | shared list: list.sort()'s in-place binarysort rewrite (no critical section) races a concurrent lock-free reader |
sort-path residual of the gh-129069 / gh-142519 list class | |
| TSAN-0025 | readline.c: set_auto_history() writes the module-global should_auto_add_history (a plain static int) unsynchronized |
belongs with the readline FT cleanup (gh-153291) | #154531 |
| TSAN-0029 | frameobject.c/sysmodule.c: trace_trampoline writes a running frame's f_trace with no critical section while the f_trace/f_trace_opcodes accessors are @critical_section |
the legacy settrace path not brought under the recent frame-accessor FT hardening; gh-116738 remit |
(low; needs mutating another thread's live frame) |
| TSAN-0034 | finalization: handle_thread_shutdown_exception reads interp->threads.head in an assert() before _PyEval_StopTheWorld, racing an exiting thread's HEAD_LOCK-held tstate_delete_common write |
— | (debug-only: the read is inside the assert; reproduced in isolation, ~44 %/run) |
Disclosure
These findings were produced with AI assistance: fusil's --tsan mode generated the concurrency stress that surfaced them, and Claude Code drafted the reports and reduced the reproducers. Every reproducer was then run and re-verified by hand on the free-threaded TSan build, and every root cause was checked against current-main source. All 15 reproduce in isolation (some probabilistically).
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
Python 3.16.0a0 free-threading build (heads/main:bcf98ddbc40, Jul 4 2026, 15:37:00) [Clang 21.1.8 (6ubuntu1)]
Linked PRs
- gh-154519
- gh-154572
- gh-154716
Hướng dẫn đóng góp
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Hướng nghiên cứu
Bắt đầu với một báo cáo TSAN chưa được nhận xử lý và đọc gist được liên kết, tệp mã nguồn được nêu tên cùng TSAN-NNNN-repro.py tối thiểu. Chạy trình tái hiện trên một bản build --disable-gil --with-thread-sanitizer với PYTHON_GIL=0, sau đó kiểm tra mã và các test liên quan. Được xem là hoàn tất khi có một issue hoặc PR CPython tập trung vào phát hiện đó và trạng thái trong issue tổng này đã được cập nhật.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- c, python
- Lĩnh vực
- compilers, testing-qa
- Loại issue
- Lỗi
- Độ khó
- 5/5
- Thời gian dự kiến
- Hơn một tuần
- Mức độ hoạt động
- Đình trệ
- Độ rõ ràng
- Khá rõ ràng
- Mức phù hợp với người mới
- 25/100