python / python/cpython

Free-threading data races in CPython: 15 findings (9 new + 6 residual of existing FT work)

Aperta
#153,852 14 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

extension-modules interpreter-core topic-free-threading type-crash
Lingua principale
Python
Stelle
77.2k
Fork
36k
Metriche PR in attesa

Descrizione

Crash report

What happened?

ThreadSanitizer fuzzing of free-threaded CPython main turned up 15 data races where a built-in object or interpreter-global state is accessed concurrently without the atomics or critical section the surrounding code (or the documentation) already assumes. Each has a minimal, stdlib-only reproducer, the raw TSan report, a root-cause analysis against current-main source, and a suggested fix, published as a self-contained gist (linked below).

They split into two groups: 9 are new defects, and 6 are residuals of free-threading work that is already merged or documented — a specific reader/field/path that a completed conversion left behind. The residuals are the lowest-risk to fix (the pattern and the fix are already in the tree next to them); the related issue/PR is named in the table.

I'm filing them under one umbrella so they can be picked off individually rather than flooding the tracker. To take one: open a normal CPython issue or PR and drop a comment here with the link — I'll mark it in the table. If any is a duplicate or a non-bug, say so and I'll annotate it.

Found with fusil's --tsan mode (fusil originally by @vstinner). Reports and reproducers were drafted with AI assistance (Claude Code) and then reviewed and re-verified by hand — see Disclosure.

Reproducing
  • Found on main (3.16.0a0) on a --disable-gil --with-thread-sanitizer debug build. These are concurrency bugs, not tied to an exact revision.
  • Each gist ships a minimal stdlib-only TSAN-NNNN-repro.py. Run it on a free-threaded TSan build with PYTHON_GIL=0; a real race exits non-zero and prints WARNING: ThreadSanitizer: data race. (TSan needs ASLR reduced — e.g. setarch -R — and an unlimited RLIMIT_AS, or it runs degraded.)
  • Most are value-benign on aligned hardware but are genuine C-level data races (formally UB, and TSan-reported); a few carry a latent UAF/leak/crash, called out per report. All 15 reproduce in isolation with a stdlib-only script (some probabilistically — loop them; the debug-only finalization assert TSAN-0034 hits ~44 %/run).
New free-threading defects (9)

Status legend: blank = not yet filed · #N = open issue/PR · #N FIXED = filed and fixed.

Report Race Suggested fix Status
TSAN-0001 multibytecodec.c: MultibyteIncrementalDecoder.getstate()/reset()/decode() on a shared decoder race its pending/pendingsize/state fields — the incremental codecs have no critical sections (incl. the iso-2022/HZ face) per-object critical sections on the incremental codec methods
TSAN-0005 _decimal.c: Decimal.__hash__ writes its lazy hash cache (self->hash) with plain stores; concurrent hash() of a shared Decimal races relaxed atomics on self->hash #154037 (PR #154045)
TSAN-0006 itertoolsmodule.c: count.__repr__ plain-reads cnt while count_next writes it with an atomic CAS — the writer was hardened, the reader missed _Py_atomic_load_ssize_relaxed in count_repr #153908 FIXED
TSAN-0011 sysmodule.c: sys.addaudithook lazily creates interp->audit_hooks with no lock, racing should_audit on every audit event — security-relevant (PEP 578); the sibling C-level hook list is already mutex-guarded serialize under runtime->audit_hooks.mutex; atomics on the pointer #154431 FIXED
TSAN-0018 dictobject.c: readers of a shared dict's dk_nentries use plain loads while setattr/insert bump it atomically — including the public PyDict_Next C-API (via _PyType_GetSubclasses), so any extension iterating a shared dict is exposed. LOAD_KEYS_NENTRIES already exists at :237 LOAD_KEYS_NENTRIES at the reader sites (_PyObject_IsInstanceDictEmpty, clear_lock_held, _PyDict_Next) #153881 FIXED
TSAN-0030 instrumentation.c: sys.monitoring.use_tool_id() is an unsynchronized check-then-act on the interpreter-global monitoring_tool_names[] — both threads pass the guard → leak + dup ownership; free_tool_id's Py_CLEAR is a UAF/double-free. All four accessors are unlocked lock/atomic the tool-id registry; fix all four accessors together #154348 (PR #154459)
TSAN-0031 _elementtree.c: concurrent feed of one shared TreeBuilder races its parse state (this/last/data/index/stack) — the module has zero critical sections, yet declares Py_MOD_GIL_NOT_USED @critical_section the whole TreeBuilder feed path (start/data/end/comment/pi/close) (abandoned PR gh-145569 covered only handle_end)
TSAN-0035 socketmodule.c: sock_timeout is read/written with plain accesses (gettimeout() vs setblocking()) — the one per-socket scalar the module's free-threading conversion (gh-128277) missed, while the sibling sock_fd and state->defaulttimeout were made atomic. _ssl.c inherits it get/set_sock_timeout over _Py_atomic_{load,store}_int64_relaxed, mirroring sock_fd #153935 (PR #153940)
TSAN-0036 instrumentation.c/ceval.c: the eval loop reads a code object's active_monitors.tools[] lock-free (no_tools_for_local_event, via gen_close) while _Py_Instrument replaces the struct under LOCK_CODE — which the eval loop never takes; lazy re-instrumentation runs with the world running relaxed atomics on the tools[] bytes (matching the file's opcode discipline) (PR gh-136994 did exactly this for the bytecode tool bytes but not active_monitors)
Residuals of existing / documented free-threading work (6)

These are a reader/field/path left behind by a completed (or documented) conversion — likely best handled as a follow-up to the named issue rather than as fresh bugs.

Report Race Related upstream Status
TSAN-0002 _zstd: ZstdCompressor.last_mode is stored plain but read lock-free via its Py_T_INT member descriptor residual of gh-133885 / gh-134289 (added locks, left last_mode plain) #154407
TSAN-0013 shared list: non-atomic readers (Py_SIZE/unpack, stringlib_bytes_join, marshal) race list_resize's atomic ob_item/ob_size publish reader-side residual of gh-129069; contract documented in gh-142519 #154516
TSAN-0014 shared list: list.sort()'s in-place binarysort rewrite (no critical section) races a concurrent lock-free reader sort-path residual of the gh-129069 / gh-142519 list class
TSAN-0025 readline.c: set_auto_history() writes the module-global should_auto_add_history (a plain static int) unsynchronized belongs with the readline FT cleanup (gh-153291) #154531
TSAN-0029 frameobject.c/sysmodule.c: trace_trampoline writes a running frame's f_trace with no critical section while the f_trace/f_trace_opcodes accessors are @critical_section the legacy settrace path not brought under the recent frame-accessor FT hardening; gh-116738 remit (low; needs mutating another thread's live frame)
TSAN-0034 finalization: handle_thread_shutdown_exception reads interp->threads.head in an assert() before _PyEval_StopTheWorld, racing an exiting thread's HEAD_LOCK-held tstate_delete_common write — (debug-only: the read is inside the assert; reproduced in isolation, ~44 %/run)
Disclosure

These findings were produced with AI assistance: fusil's --tsan mode generated the concurrency stress that surfaced them, and Claude Code drafted the reports and reduced the reproducers. Every reproducer was then run and re-verified by hand on the free-threaded TSan build, and every root cause was checked against current-main source. All 15 reproduce in isolation (some probabilistically).

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.16.0a0 free-threading build (heads/main:bcf98ddbc40, Jul 4 2026, 15:37:00) [Clang 21.1.8 (6ubuntu1)]

Linked PRs
  • gh-154519
  • gh-154572
  • gh-154716

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Direzione di ricerca

Inizia con un report TSAN non assegnato e leggi il gist collegato, il file sorgente indicato e il TSAN-NNNN-repro.py minimale. Esegui il riproduttore su una build --disable-gil --with-thread-sanitizer con PYTHON_GIL=0, quindi esamina il codice e i test correlati. Il lavoro è completato quando esiste un issue o PR di CPython mirato a quella scoperta e lo stato è stato aggiornato in questo issue ombrello.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
c, python
Ambito
compilers, testing-qa
Tipo di issue
Bug
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.