python / python/cpython

Free-threading data races in CPython: 15 findings (9 new + 6 residual of existing FT work)

Ouverte
#153,852 14 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

extension-modules interpreter-core topic-free-threading type-crash
Langage dominant
Python
Étoiles
77.2k
Forks
35.9k
Métriques de PR en attente

Description

Crash report

What happened?

ThreadSanitizer fuzzing of free-threaded CPython main turned up 15 data races where a built-in object or interpreter-global state is accessed concurrently without the atomics or critical section the surrounding code (or the documentation) already assumes. Each has a minimal, stdlib-only reproducer, the raw TSan report, a root-cause analysis against current-main source, and a suggested fix, published as a self-contained gist (linked below).

They split into two groups: 9 are new defects, and 6 are residuals of free-threading work that is already merged or documented — a specific reader/field/path that a completed conversion left behind. The residuals are the lowest-risk to fix (the pattern and the fix are already in the tree next to them); the related issue/PR is named in the table.

I'm filing them under one umbrella so they can be picked off individually rather than flooding the tracker. To take one: open a normal CPython issue or PR and drop a comment here with the link — I'll mark it in the table. If any is a duplicate or a non-bug, say so and I'll annotate it.

Found with fusil's --tsan mode (fusil originally by @vstinner). Reports and reproducers were drafted with AI assistance (Claude Code) and then reviewed and re-verified by hand — see Disclosure.

Reproducing
  • Found on main (3.16.0a0) on a --disable-gil --with-thread-sanitizer debug build. These are concurrency bugs, not tied to an exact revision.
  • Each gist ships a minimal stdlib-only TSAN-NNNN-repro.py. Run it on a free-threaded TSan build with PYTHON_GIL=0; a real race exits non-zero and prints WARNING: ThreadSanitizer: data race. (TSan needs ASLR reduced — e.g. setarch -R — and an unlimited RLIMIT_AS, or it runs degraded.)
  • Most are value-benign on aligned hardware but are genuine C-level data races (formally UB, and TSan-reported); a few carry a latent UAF/leak/crash, called out per report. All 15 reproduce in isolation with a stdlib-only script (some probabilistically — loop them; the debug-only finalization assert TSAN-0034 hits ~44 %/run).
New free-threading defects (9)

Status legend: blank = not yet filed · #N = open issue/PR · #N FIXED = filed and fixed.

Report Race Suggested fix Status
TSAN-0001 multibytecodec.c: MultibyteIncrementalDecoder.getstate()/reset()/decode() on a shared decoder race its pending/pendingsize/state fields — the incremental codecs have no critical sections (incl. the iso-2022/HZ face) per-object critical sections on the incremental codec methods
TSAN-0005 _decimal.c: Decimal.__hash__ writes its lazy hash cache (self->hash) with plain stores; concurrent hash() of a shared Decimal races relaxed atomics on self->hash #154037 (PR #154045)
TSAN-0006 itertoolsmodule.c: count.__repr__ plain-reads cnt while count_next writes it with an atomic CAS — the writer was hardened, the reader missed _Py_atomic_load_ssize_relaxed in count_repr #153908 FIXED
TSAN-0011 sysmodule.c: sys.addaudithook lazily creates interp->audit_hooks with no lock, racing should_audit on every audit event — security-relevant (PEP 578); the sibling C-level hook list is already mutex-guarded serialize under runtime->audit_hooks.mutex; atomics on the pointer #154431 FIXED
TSAN-0018 dictobject.c: readers of a shared dict's dk_nentries use plain loads while setattr/insert bump it atomically — including the public PyDict_Next C-API (via _PyType_GetSubclasses), so any extension iterating a shared dict is exposed. LOAD_KEYS_NENTRIES already exists at :237 LOAD_KEYS_NENTRIES at the reader sites (_PyObject_IsInstanceDictEmpty, clear_lock_held, _PyDict_Next) #153881 FIXED
TSAN-0030 instrumentation.c: sys.monitoring.use_tool_id() is an unsynchronized check-then-act on the interpreter-global monitoring_tool_names[] — both threads pass the guard → leak + dup ownership; free_tool_id's Py_CLEAR is a UAF/double-free. All four accessors are unlocked lock/atomic the tool-id registry; fix all four accessors together #154348 (PR #154459)
TSAN-0031 _elementtree.c: concurrent feed of one shared TreeBuilder races its parse state (this/last/data/index/stack) — the module has zero critical sections, yet declares Py_MOD_GIL_NOT_USED @critical_section the whole TreeBuilder feed path (start/data/end/comment/pi/close) (abandoned PR gh-145569 covered only handle_end)
TSAN-0035 socketmodule.c: sock_timeout is read/written with plain accesses (gettimeout() vs setblocking()) — the one per-socket scalar the module's free-threading conversion (gh-128277) missed, while the sibling sock_fd and state->defaulttimeout were made atomic. _ssl.c inherits it get/set_sock_timeout over _Py_atomic_{load,store}_int64_relaxed, mirroring sock_fd #153935 (PR #153940)
TSAN-0036 instrumentation.c/ceval.c: the eval loop reads a code object's active_monitors.tools[] lock-free (no_tools_for_local_event, via gen_close) while _Py_Instrument replaces the struct under LOCK_CODE — which the eval loop never takes; lazy re-instrumentation runs with the world running relaxed atomics on the tools[] bytes (matching the file's opcode discipline) (PR gh-136994 did exactly this for the bytecode tool bytes but not active_monitors)
Residuals of existing / documented free-threading work (6)

These are a reader/field/path left behind by a completed (or documented) conversion — likely best handled as a follow-up to the named issue rather than as fresh bugs.

Report Race Related upstream Status
TSAN-0002 _zstd: ZstdCompressor.last_mode is stored plain but read lock-free via its Py_T_INT member descriptor residual of gh-133885 / gh-134289 (added locks, left last_mode plain) #154407
TSAN-0013 shared list: non-atomic readers (Py_SIZE/unpack, stringlib_bytes_join, marshal) race list_resize's atomic ob_item/ob_size publish reader-side residual of gh-129069; contract documented in gh-142519 #154516
TSAN-0014 shared list: list.sort()'s in-place binarysort rewrite (no critical section) races a concurrent lock-free reader sort-path residual of the gh-129069 / gh-142519 list class
TSAN-0025 readline.c: set_auto_history() writes the module-global should_auto_add_history (a plain static int) unsynchronized belongs with the readline FT cleanup (gh-153291) #154531
TSAN-0029 frameobject.c/sysmodule.c: trace_trampoline writes a running frame's f_trace with no critical section while the f_trace/f_trace_opcodes accessors are @critical_section the legacy settrace path not brought under the recent frame-accessor FT hardening; gh-116738 remit (low; needs mutating another thread's live frame)
TSAN-0034 finalization: handle_thread_shutdown_exception reads interp->threads.head in an assert() before _PyEval_StopTheWorld, racing an exiting thread's HEAD_LOCK-held tstate_delete_common write — (debug-only: the read is inside the assert; reproduced in isolation, ~44 %/run)
Disclosure

These findings were produced with AI assistance: fusil's --tsan mode generated the concurrency stress that surfaced them, and Claude Code drafted the reports and reduced the reproducers. Every reproducer was then run and re-verified by hand on the free-threaded TSan build, and every root cause was checked against current-main source. All 15 reproduce in isolation (some probabilistically).

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.16.0a0 free-threading build (heads/main:bcf98ddbc40, Jul 4 2026, 15:37:00) [Clang 21.1.8 (6ubuntu1)]

Linked PRs
  • gh-154519
  • gh-154572
  • gh-154716

Guide de contribution

Ouvrir le guide de contribution

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Piste de recherche

Commencez par un rapport TSAN non revendiqué et lisez le gist qui lui est lié, le fichier source indiqué et le TSAN-NNNN-repro.py minimal. Exécutez le reproducteur avec une build --disable-gil --with-thread-sanitizer et PYTHON_GIL=0, puis examinez le code et les tests concernés. C’est terminé lorsqu’un issue ou PR CPython ciblé a été créé pour cette découverte et que le statut a été mis à jour dans cet issue chapeau.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
c, python
Domaine
compilers, testing-qa
Type d'issue
Bug
Difficulté
5/5
Temps estimé
Plus d'une semaine
Activité
À l'abandon
Clarté
Plutôt claire
Accessibilité débutants
25/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.