Free-threaded CPython 3.14: concurrent functools.partial.__setstate__ crashes with repr(), calls, or another __setstate__

Prior report

gh-157099 (2026-09-07) reported the same missing synchronization in partial_setstate, reaching it through partial_new (a crash in tuple_concat under ASan, 3.14.7 @ 823f032). It was closed by its reporter after a request to consolidate a batch of reports (gh-157124), and the consolidated issues gh-157196 / gh-157197 / gh-157198 were then withdrawn by the reporter on 2026-09-11 to be resubmitted under the security policy. As far as I can see nothing tracks it now, and main @ 89c67a9 still crashes. This report was found independently (static analysis of partial_repr on 3.14.0rc1, then runtime confirmation) and adds a sanitizer-free 25-line reproducer covering repr(), calls and writer/writer, with a GIL control on the same binary. If you would rather reopen gh-157099, please close this one as a duplicate. Filed on the public tracker because triggering it requires running Python code in the process (the assessment given on gh-157196).

What happened?

On the free-threaded build, calling partial.__setstate__ on a functools.partial object while another thread uses the same object (repr(p) or p()), or while a second thread also calls __setstate__, crashes the interpreter within seconds. The read-only control completed all five four-second runs without a crash. The same binary with PYTHON_GIL=1 completed every run without a crash.

Standalone reproducer (repro_partial.py <mode>):

import functools, itertools, os, sys, threading
if os.environ.get("ALLOW_GIL") != "1":
    assert not sys._is_gil_enabled()
mode = sys.argv[1]
counter = itertools.count(10**12)          # fresh heap ints, never cached
def fresh(): return next(counter)
def f(*a, **k): return None
p = functools.partial(f, fresh(), fresh())
stop = threading.Event()
def reader_repr(): repr(p)
def reader_call(): p()
def writer(): p.__setstate__((f, (fresh(), fresh()), {}, None))
reader, nr, nw = {"ro": (reader_repr, 4, 0), "rw": (reader_repr, 4, 1),
                  "ww": (None, 0, 2), "cw": (reader_call, 4, 1)}[mode]
def loop(fn):
    while not stop.is_set():
        fn()
ts = [threading.Thread(target=loop, args=(reader,)) for _ in range(nr)] + \
     [threading.Thread(target=loop, args=(writer,)) for _ in range(nw)]
for t in ts: t.start()
stop.wait(4); stop.set()
for t in ts: t.join()
print("ok")

Results, 5 runs × 4 s per mode, run with -X faulthandler ("hang" = the process had not exited 30 s after start and was killed):

mode	threads	3.14.6t	3.14.6t, PYTHONMALLOC=debug	same binary, PYTHON_GIL=1	main @ 89c67a9 (--disable-gil)	main, PYTHONMALLOC=debug
ro	4 × repr(p)	5/5 ok	5/5 ok	—	5/5 ok	5/5 ok
rw	4 × repr(p) + 1 × __setstate__	0/5 (SIGILL, SIGABRT, 3 hangs)	0/5 (SIGSEGV ×5)	3/3 ok	0/5 (SIGABRT ×2, SIGILL, 2 hangs)	0/5 (SIGSEGV ×5)
ww	2 × __setstate__	4/5 (SIGSEGV ×1)	1/5 (SIGSEGV ×4)	3/3 ok	2/5 (SIGSEGV ×3)	4/5 (SIGSEGV ×1)
cw	4 × p() + 1 × __setstate__	0/5 (SIGILL ×2, SIGTRAP ×2, SIGSEGV)	0/5 (SIGSEGV ×5)	3/3 ok	0/5 (SIGSEGV ×3, SIGILL, 1 hang)	0/5 (SIGSEGV ×5)

ww is intermittent: over several runs of this script today it crashed in 1 to 4 of 5 runs per configuration. rw and cw failed in every run of every free-threaded configuration. The SIGABRT cases are

Fatal Python error: PyMutex_Unlock: unlocking mutex that is not locked

raised from the repr thread (faulthandler shows reader_repr). The repr/__setstate__ shape also crashes on 3.14.0rc1 free-threaded (5/5, PYTHONMALLOC=debug). Note that main already carries the partial_vectorcall snapshot from gh-154189, and cw still crashes there.

Analysis

partial_setstate replaces pto->fn, pto->args and pto->kw with Py_SETREF and holds no critical section on the partial object (3.13 branch Modules/_functoolsmodule.c:514; 3.14 branch :762; main :857). The readers take strong references from the fields — partial_repr since gh-144475 (3.14 branch :630), partial_vectorcall since gh-154189 (main only; not in 3.14). The source suggests an unsynchronized reference-acquisition gap: Py_NewRef(pto->args) does not protect the interval between loading the field and acquiring a strong reference while another thread replaces it. The observed crashes have not yet been attributed to a particular native access. Two concurrent __setstate__ writers also crash without a reader thread; the failing access in that case has not yet been identified. gh-144475 and gh-154189 addressed the re-entrant, same-thread shape of this; gh-157099 and this report are the cross-thread shape.

Expected behaviour

I expect concurrent use to avoid interpreter crashes; no particular ordering or consistent result is required by this reproducer. The free-threading HOWTO says the build "aims to provide similar thread-safety behavior at the Python level to the default GIL-enabled build" while noting that this "should be treated as a description of the current implementation, not a guarantee"; the same binary with the GIL enabled runs the reproducer without a crash.

Possible approach

A possible approach is to synchronize state publication and acquisition of coherent reader snapshots on the partial object, including associated placeholder and vectorcall state; the complete access and reentrancy audit remains to be done. Happy to test a patch.

CPython versions tested on

3.14.6 (free-threaded, python-build-standalone via uv), 3.14.0rc1 (free-threaded), CPython main branch @ 89c67a9 (2026-09-20, local --disable-gil build; Modules/_functoolsmodule.c identical to upstream at that commit)

Operating systems tested on

macOS (arm64, Darwin 25.6)

Output from running 'python -VV' on the command line

Python 3.14.6 free-threading build (main, Jul 18 2026, 17:07:52) [Clang 22.1.3 ]
Python 3.16.0a0 free-threading build (remotes/origin/HEAD-dirty:89c67a9, Sep 20 2026, 14:12:46) [Clang 21.0.0 (clang-2100.3.34.2)]

Related

gh-157099 (same root cause, closed not_planned), gh-144475, gh-154189, gh-140590

從 repro_partial.py 中的獨立重現開始，在 free-threaded 建置上執行其 ro、rw、ww 和 cw 模式，並使用 PYTHON_GIL=1 作為對照。接著檢查 Modules/_functoolsmodule.c 中的 partial_setstate、partial_repr 和 partial_vectorcall，包括報告中引用的相關變更。當並行使用不再導致直譯器崩潰，同時維持一致的狀態存取時，即視為完成。