python / python/cpython

stale `MemoryError` trips `_Py_CheckSlotResult` in `reload_singlephase_extension`

Open
#153,797 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

interpreter-core type-crash
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Crash report

What happened?

This is sub-issue of https://github.com/python/cpython/issues/151763
When _modules_by_index_set() fails under OOM, the cleanup PyMapping_DelItem(modules, name) runs the dict __delitem__ slot with the MemoryError still set; the delete succeeds with an exception pending, so the debug _Py_CheckSlotResult assert fatals the interpreter.

AI Disclaimer: this gist was drafted by Claude Code, which also generated the reduced reproducer.

Crash report

Re-importing a single-phase (legacy) C extension module routes through reload_singlephase_extension(). After the module is rebuilt it calls _modules_by_index_set() (Python/import.c:2010) to record it in the per-interpreter modules_by_index list. Under OOM that call fails (its internal PyList_New/PyList_Append raises MemoryError and returns -1). The error-cleanup branch then runs PyMapping_DelItem(modules, info->name) (import.c:2011) with the MemoryError still set. The delete from sys.modules (a dict) succeeds, so the debug check assert(_Py_CheckSlotResult(o, "__delitem__", res >= 0)) (Objects/abstract.c:280) sees success && PyErr_Occurred() and aborts the interpreter via _Py_FatalErrorFormat.

Reproducer

Minimal, stdlib-only (shrinkray-reduced from the vehicle, then cleaned; deterministic on the
ft_debug_asan build, re-verified 10/10). A dict __delitem__ slot succeeds with an
exception set under OOM. (This minimal repro is ft_debug_asan-specific; the build matrix
below is verified against the full vehicle_source.py, which also fatals on the jit debug
build via the readline single-phase-extension reload path the minimal form doesn't exercise.)

import faulthandler, pdb
faulthandler.enable()
from _testcapi import set_nomemory
for start in range(1000):
    set_nomemory(start)
    try:
        pdb.runcall()
    except BaseException:
        pass
print("done, no crash")

The full fuzzer vehicle is preserved as vehicle_source.py.

Backtrace

#8  _Py_CheckSlotResult          Objects/call.c:80      <- "Slot __delitem__ ... succeeded with an exception set"
#9  PyObject_DelItem             Objects/abstract.c:280 <- assert(_Py_CheckSlotResult(o,"__delitem__",res>=0))
#10 reload_singlephase_extension Python/import.c:2011   <- PyMapping_DelItem(modules, info->name) cleanup
#11 import_find_extension        Python/import.c:2043
#12 _imp_create_dynamic_impl     Python/import.c:5468

The faulting object is the live sys.modules dict and the delete succeeds
(res == 0); the pending object is a MemoryError (refcount 1) left set by the
failed _modules_by_index_set() at import.c:2010. This is a stale-exception /
missing-error-clear bug, not a NULL/freed pointer.

Root cause

Python/import.c, reload_singlephase_extension() (L2009-2014):

    Py_ssize_t index = _get_cached_module_index(cached);
    if (_modules_by_index_set(tstate->interp, index, mod) < 0) {   /* L2010: raises MemoryError under OOM */
        PyMapping_DelItem(modules, info->name);                    /* L2011: runs with MemoryError still set */
        Py_DECREF(mod);
        return NULL;
    }

_modules_by_index_set() (L577) can fail by raising MemoryError:

    if (MODULES_BY_INDEX(interp) == NULL) {
        MODULES_BY_INDEX(interp) = PyList_New(0);          /* alloc -> can fail */
        ...
    }
    while (PyList_GET_SIZE(MODULES_BY_INDEX(interp)) <= index) {
        if (PyList_Append(MODULES_BY_INDEX(interp), Py_None) < 0) {   /* alloc -> can fail */
            return -1;
        }
    }

When it returns -1 with MemoryError set, the cleanup calls
PyMapping_DelItem(modules, info->name) -> PyObject_DelItem ->
dict_ass_subscript (sys.modules is a dict). The key is present, so the slot
succeeds (res == 0). On a debug build, PyObject_DelItem
(Objects/abstract.c:280) wraps the call in
assert(_Py_CheckSlotResult(o, "__delitem__", res >= 0)), and
_Py_CheckSlotResult (Objects/call.c:86-90) treats "slot succeeded while an
exception is set" as a fatal interpreter invariant violation, calling
_Py_FatalErrorFormat. The defect is that the cleanup path performs an operation
that runs the dict __delitem__ slot without first preserving/clearing the
already-pending exception.

Suggested fix

Use the existing remove_module() cleanup helper instead of calling
PyMapping_DelItem(modules, info->name) directly in the
_modules_by_index_set() failure path:

    if (_modules_by_index_set(tstate->interp, index, mod) < 0) {
        remove_module(tstate, info->name);
        Py_DECREF(mod);
        return NULL;
    }
CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Output from running 'python -VV' on the command line:

No response

Linked PRs
  • gh-153798

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in Python/import.c at reload_singlephase_extension() around lines 2009-2014, then read the remove_module() helper and _modules_by_index_set(). Run the provided minimal reproducer on an ft_debug_asan build; done means the OOM cleanup no longer triggers _Py_CheckSlotResult or fatals the interpreter.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.