Free-threaded CPython 3.14: concurrent functools.partial.__setstate__ crashes with repr(), calls, or another __setstate__

Prior report

gh-157099 (2026-09-07) reported the same missing synchronization in partial_setstate, reaching it through partial_new (a crash in tuple_concat under ASan, 3.14.7 @ 823f032). It was closed by its reporter after a request to consolidate a batch of reports (gh-157124), and the consolidated issues gh-157196 / gh-157197 / gh-157198 were then withdrawn by the reporter on 2026-09-11 to be resubmitted under the security policy. As far as I can see nothing tracks it now, and main @ 89c67a9 still crashes. This report was found independently (static analysis of partial_repr on 3.14.0rc1, then runtime confirmation) and adds a sanitizer-free 25-line reproducer covering repr(), calls and writer/writer, with a GIL control on the same binary. If you would rather reopen gh-157099, please close this one as a duplicate. Filed on the public tracker because triggering it requires running Python code in the process (the assessment given on gh-157196).

What happened?

On the free-threaded build, calling partial.__setstate__ on a functools.partial object while another thread uses the same object (repr(p) or p()), or while a second thread also calls __setstate__, crashes the interpreter within seconds. The read-only control completed all five four-second runs without a crash. The same binary with PYTHON_GIL=1 completed every run without a crash.

Standalone reproducer (repro_partial.py <mode>):

import functools, itertools, os, sys, threading
if os.environ.get("ALLOW_GIL") != "1":
    assert not sys._is_gil_enabled()
mode = sys.argv[1]
counter = itertools.count(10**12)          # fresh heap ints, never cached
def fresh(): return next(counter)
def f(*a, **k): return None
p = functools.partial(f, fresh(), fresh())
stop = threading.Event()
def reader_repr(): repr(p)
def reader_call(): p()
def writer(): p.__setstate__((f, (fresh(), fresh()), {}, None))
reader, nr, nw = {"ro": (reader_repr, 4, 0), "rw": (reader_repr, 4, 1),
                  "ww": (None, 0, 2), "cw": (reader_call, 4, 1)}[mode]
def loop(fn):
    while not stop.is_set():
        fn()
ts = [threading.Thread(target=loop, args=(reader,)) for _ in range(nr)] + \
     [threading.Thread(target=loop, args=(writer,)) for _ in range(nw)]
for t in ts: t.start()
stop.wait(4); stop.set()
for t in ts: t.join()
print("ok")

Results, 5 runs × 4 s per mode, run with -X faulthandler ("hang" = the process had not exited 30 s after start and was killed):

mode	threads	3.14.6t	3.14.6t, PYTHONMALLOC=debug	same binary, PYTHON_GIL=1	main @ 89c67a9 (--disable-gil)	main, PYTHONMALLOC=debug
ro	4 × repr(p)	5/5 ok	5/5 ok	—	5/5 ok	5/5 ok
rw	4 × repr(p) + 1 × __setstate__	0/5 (SIGILL, SIGABRT, 3 hangs)	0/5 (SIGSEGV ×5)	3/3 ok	0/5 (SIGABRT ×2, SIGILL, 2 hangs)	0/5 (SIGSEGV ×5)
ww	2 × __setstate__	4/5 (SIGSEGV ×1)	1/5 (SIGSEGV ×4)	3/3 ok	2/5 (SIGSEGV ×3)	4/5 (SIGSEGV ×1)
cw	4 × p() + 1 × __setstate__	0/5 (SIGILL ×2, SIGTRAP ×2, SIGSEGV)	0/5 (SIGSEGV ×5)	3/3 ok	0/5 (SIGSEGV ×3, SIGILL, 1 hang)	0/5 (SIGSEGV ×5)

ww is intermittent: over several runs of this script today it crashed in 1 to 4 of 5 runs per configuration. rw and cw failed in every run of every free-threaded configuration. The SIGABRT cases are

Fatal Python error: PyMutex_Unlock: unlocking mutex that is not locked

raised from the repr thread (faulthandler shows reader_repr). The repr/__setstate__ shape also crashes on 3.14.0rc1 free-threaded (5/5, PYTHONMALLOC=debug). Note that main already carries the partial_vectorcall snapshot from gh-154189, and cw still crashes there.

Analysis

partial_setstate replaces pto->fn, pto->args and pto->kw with Py_SETREF and holds no critical section on the partial object (3.13 branch Modules/_functoolsmodule.c:514; 3.14 branch :762; main :857). The readers take strong references from the fields — partial_repr since gh-144475 (3.14 branch :630), partial_vectorcall since gh-154189 (main only; not in 3.14). The source suggests an unsynchronized reference-acquisition gap: Py_NewRef(pto->args) does not protect the interval between loading the field and acquiring a strong reference while another thread replaces it. The observed crashes have not yet been attributed to a particular native access. Two concurrent __setstate__ writers also crash without a reader thread; the failing access in that case has not yet been identified. gh-144475 and gh-154189 addressed the re-entrant, same-thread shape of this; gh-157099 and this report are the cross-thread shape.

Expected behaviour

I expect concurrent use to avoid interpreter crashes; no particular ordering or consistent result is required by this reproducer. The free-threading HOWTO says the build "aims to provide similar thread-safety behavior at the Python level to the default GIL-enabled build" while noting that this "should be treated as a description of the current implementation, not a guarantee"; the same binary with the GIL enabled runs the reproducer without a crash.

Possible approach

A possible approach is to synchronize state publication and acquisition of coherent reader snapshots on the partial object, including associated placeholder and vectorcall state; the complete access and reentrancy audit remains to be done. Happy to test a patch.

CPython versions tested on

3.14.6 (free-threaded, python-build-standalone via uv), 3.14.0rc1 (free-threaded), CPython main branch @ 89c67a9 (2026-09-20, local --disable-gil build; Modules/_functoolsmodule.c identical to upstream at that commit)

Operating systems tested on

macOS (arm64, Darwin 25.6)

Output from running 'python -VV' on the command line

Python 3.14.6 free-threading build (main, Jul 18 2026, 17:07:52) [Clang 22.1.3 ]
Python 3.16.0a0 free-threading build (remotes/origin/HEAD-dirty:89c67a9, Sep 20 2026, 14:12:46) [Clang 21.0.0 (clang-2100.3.34.2)]

Related

gh-157099 (same root cause, closed not_planned), gh-144475, gh-154189, gh-140590

Bắt đầu với bản tái hiện độc lập trong repro_partial.py, chạy các chế độ ro, rw, ww và cw của nó trên một bản build free-threaded và với PYTHON_GIL=1 làm đối chứng. Sau đó kiểm tra partial_setstate, partial_repr và partial_vectorcall trong Modules/_functoolsmodule.c, bao gồm các thay đổi liên quan được trích dẫn trong báo cáo. Công việc được xem là hoàn tất khi việc sử dụng đồng thời không còn làm trình thông dịch bị crash, đồng thời vẫn duy trì quyền truy cập trạng thái nhất quán.