Sharing a `contextvars.Context` iterator across threads crashes (HAMT iterator cursor corruption) under free-threading

Crash report

What happened?

On the free-threaded build, advancing a single shared iterator over a contextvars.Context from multiple threads corrupts the iterator's traversal cursor and dereferences a garbage node pointer → SIGSEGV.

The HAMT iterator keeps its whole cursor — i_nodes[] (borrowed node pointers), i_pos[], i_level — inside the iterator object and advances it with plain reads/writes and no critical section (Python/hamt.c, main@a1d580430c8):

static hamt_iter_t
hamt_iterator_next(PyHamtIteratorState *iter, PyObject **key, PyObject **val) {
    if (iter->i_level < 0)                       /* :2188  plain read of the cursor level */
        return I_END;
    PyHamtNode *current = iter->i_nodes[iter->i_level];   /* :2194  read node at current level */
    if (IS_BITMAP_NODE(current)) {               /* :2196  Py_TYPE(current) -> SEGV on a wild node */
        return hamt_iterator_bitmap_next(iter, key, val);
    }
    ...
}

static hamt_iter_t
hamt_iterator_bitmap_next(PyHamtIteratorState *iter, ...) {
    int8_t level = iter->i_level;
    PyHamtNode_Bitmap *node = (PyHamtNode_Bitmap *)(iter->i_nodes[level]);
    if (pos + 1 >= Py_SIZE(node)) {              /* :2092  Py_SIZE(node) -> SEGV on a wild node */
        iter->i_level--;                         /* :2097  plain write of the cursor level */
        return hamt_iterator_next(iter, key, val);
    }
    ...
}

Two threads calling next() on the same iterator desync i_level against i_nodes[], so a thread reads current = iter->i_nodes[iter->i_level] as a stale / NULL / wild pointer and immediately evaluates IS_BITMAP_NODE(current) = Py_TYPE(current) (or Py_SIZE(node) in hamt_iterator_bitmap_next) — dereferencing garbage. Because the nodes in i_nodes are held borrowed (hamt_iterator_init: "we don't incref/decref nodes in i_nodes"), a concurrent structural change is not even required — cursor desync alone produces the wild dereference. This is safe under the GIL (one thread runs the advance at a time).

The HAMT iterator backs contextvars.Context iteration — iter(ctx), ctx.keys(), ctx.values(), ctx.items() — via hamt_baseiter_tp_iternext (hamt.c:2479).

Reproducer

Free-threaded build, PYTHON_GIL=0:

import contextvars, threading
NT = 8; ITERS = 40000
vs = [contextvars.ContextVar(f"v{i}") for i in range(16)]
ctx = contextvars.copy_context()
def populate():
    for i, v in enumerate(vs):
        v.set(i)
ctx.run(populate)
cell = [iter(ctx)]
def worker():
    for _ in range(ITERS):
        it = cell[0]
        try: next(it)
        except StopIteration: cell[0] = iter(ctx)
        except Exception: pass
ts = [threading.Thread(target=worker) for _ in range(NT)]
for t in ts: t.start()
for t in ts: t.join()

• 6/6 SIGSEGV on a plain free-threaded debug build (no sanitizer).
• On a free-threaded ASan build:

AddressSanitizer: SEGV in _Py_TYPE_impl  Include/object.h:234
  hamt_iterator_bitmap_next  Python/hamt.c:2092
  hamt_iterator_next         Python/hamt.c:2196
  hamt_baseiter_tp_iternext  Python/hamt.c:2479
  context_run                Python/context.c:731

• Under TSan: WARNING: ThreadSanitizer: data race … Python/hamt.c:2188 in hamt_iterator_next.

The crash is not Py_DEBUG-only — it reproduces as a raw SIGSEGV from the wild-pointer dereference on the plain free-threaded build.

Relationship to gh-124397

Per gh-124397 ("Strategy for Iterators in Free Threading") point 3, C iterators should get "the minimal changes necessary to cause them to not crash … Concurrent access is allowed to return duplicate values, skip values, or raise an exception." The HAMT / Context iterator was never brought under that hardening, so it crashes rather than returning dup/skip. It is the contextvars.Context sibling of the already-tracked shared-iterator crashes: dict (#154130), set (#144357), and memoryview.

Suggested fix

Bring the HAMT iterator advance under a per-iterator critical section (or make the i_level/i_pos/i_nodes cursor accesses atomic with a NULL/bounds check on the borrowed i_nodes[i_level] before dereferencing), matching the dictiter/setiter hardening. The guard must span the whole hamt_iterator_next → *_bitmap_next / *_array_next descent because i_nodes holds borrowed pointers.

Found with fusil's --tsan mode; reproducer reduced and this report drafted with AI assistance (Claude Code), then verified by hand on free-threaded debug / ASan / TSan builds of main.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.16.0a0 free-threading build (heads/main:a1d580430c8, Jul 18 2026, 20:23:36) [Clang 21.1.8 (6ubuntu1)]

Linked PRs

• gh-154586

Bắt đầu trong Python/hamt.c, tại hamt_iterator_next, hamt_iterator_bitmap_next và hamt_baseiter_tp_iternext, sau đó tái hiện lỗi bằng free-threaded Context iterator script được cung cấp. So sánh với hardening được mô tả cho các iterator của dict và set. Được xem là hoàn tất khi việc tiến hành đồng thời không còn gây crash hoặc dereference các borrowed nodes không hợp lệ, và các free-threaded checks liên quan đều vượt qua.