stale `MemoryError` trips `_Py_CheckSlotResult` in `reload_singlephase_extension`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Crash report
What happened?
This is sub-issue of https://github.com/python/cpython/issues/151763
When _modules_by_index_set() fails under OOM, the cleanup PyMapping_DelItem(modules, name) runs the dict __delitem__ slot with the MemoryError still set; the delete succeeds with an exception pending, so the debug _Py_CheckSlotResult assert fatals the interpreter.
AI Disclaimer: this gist was drafted by Claude Code, which also generated the reduced reproducer.
Crash report
Re-importing a single-phase (legacy) C extension module routes through reload_singlephase_extension(). After the module is rebuilt it calls _modules_by_index_set() (Python/import.c:2010) to record it in the per-interpreter modules_by_index list. Under OOM that call fails (its internal PyList_New/PyList_Append raises MemoryError and returns -1). The error-cleanup branch then runs PyMapping_DelItem(modules, info->name) (import.c:2011) with the MemoryError still set. The delete from sys.modules (a dict) succeeds, so the debug check assert(_Py_CheckSlotResult(o, "__delitem__", res >= 0)) (Objects/abstract.c:280) sees success && PyErr_Occurred() and aborts the interpreter via _Py_FatalErrorFormat.
Reproducer
Minimal, stdlib-only (shrinkray-reduced from the vehicle, then cleaned; deterministic on the
ft_debug_asan build, re-verified 10/10). A dict __delitem__ slot succeeds with an
exception set under OOM. (This minimal repro is ft_debug_asan-specific; the build matrix
below is verified against the full vehicle_source.py, which also fatals on the jit debug
build via the readline single-phase-extension reload path the minimal form doesn't exercise.)
import faulthandler, pdb
faulthandler.enable()
from _testcapi import set_nomemory
for start in range(1000):
set_nomemory(start)
try:
pdb.runcall()
except BaseException:
pass
print("done, no crash")
The full fuzzer vehicle is preserved as vehicle_source.py.
Backtrace
#8 _Py_CheckSlotResult Objects/call.c:80 <- "Slot __delitem__ ... succeeded with an exception set"
#9 PyObject_DelItem Objects/abstract.c:280 <- assert(_Py_CheckSlotResult(o,"__delitem__",res>=0))
#10 reload_singlephase_extension Python/import.c:2011 <- PyMapping_DelItem(modules, info->name) cleanup
#11 import_find_extension Python/import.c:2043
#12 _imp_create_dynamic_impl Python/import.c:5468
The faulting object is the live sys.modules dict and the delete succeeds
(res == 0); the pending object is a MemoryError (refcount 1) left set by the
failed _modules_by_index_set() at import.c:2010. This is a stale-exception /
missing-error-clear bug, not a NULL/freed pointer.
Root cause
Python/import.c, reload_singlephase_extension() (L2009-2014):
Py_ssize_t index = _get_cached_module_index(cached);
if (_modules_by_index_set(tstate->interp, index, mod) < 0) { /* L2010: raises MemoryError under OOM */
PyMapping_DelItem(modules, info->name); /* L2011: runs with MemoryError still set */
Py_DECREF(mod);
return NULL;
}
_modules_by_index_set() (L577) can fail by raising MemoryError:
if (MODULES_BY_INDEX(interp) == NULL) {
MODULES_BY_INDEX(interp) = PyList_New(0); /* alloc -> can fail */
...
}
while (PyList_GET_SIZE(MODULES_BY_INDEX(interp)) <= index) {
if (PyList_Append(MODULES_BY_INDEX(interp), Py_None) < 0) { /* alloc -> can fail */
return -1;
}
}
When it returns -1 with MemoryError set, the cleanup calls
PyMapping_DelItem(modules, info->name) -> PyObject_DelItem ->
dict_ass_subscript (sys.modules is a dict). The key is present, so the slot
succeeds (res == 0). On a debug build, PyObject_DelItem
(Objects/abstract.c:280) wraps the call in
assert(_Py_CheckSlotResult(o, "__delitem__", res >= 0)), and
_Py_CheckSlotResult (Objects/call.c:86-90) treats "slot succeeded while an
exception is set" as a fatal interpreter invariant violation, calling
_Py_FatalErrorFormat. The defect is that the cleanup path performs an operation
that runs the dict __delitem__ slot without first preserving/clearing the
already-pending exception.
Suggested fix
Use the existing remove_module() cleanup helper instead of calling
PyMapping_DelItem(modules, info->name) directly in the
_modules_by_index_set() failure path:
if (_modules_by_index_set(tstate->interp, index, mod) < 0) {
remove_module(tstate, info->name);
Py_DECREF(mod);
return NULL;
}
CPython versions tested on:
CPython main branch
Operating systems tested on:
No response
Output from running 'python -VV' on the command line:
No response
Linked PRs
- gh-153798
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in Python/import.c at reload_singlephase_extension() around lines 2009-2014, then read the remove_module() helper and _modules_by_index_set(). Run the provided minimal reproducer on an ft_debug_asan build; done means the OOM cleanup no longer triggers _Py_CheckSlotResult or fatals the interpreter.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100