python / python/cpython

stale `MemoryError` trips `_Py_CheckSlotResult` in `reload_singlephase_extension`

未关闭
#153,797 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

interpreter-core type-crash
主要语言
Python
星标
77.2k
派生
35.9k
PR 合并指标
PR 指标待抓取

描述

Crash report

What happened?

This is sub-issue of https://github.com/python/cpython/issues/151763
When _modules_by_index_set() fails under OOM, the cleanup PyMapping_DelItem(modules, name) runs the dict __delitem__ slot with the MemoryError still set; the delete succeeds with an exception pending, so the debug _Py_CheckSlotResult assert fatals the interpreter.

AI Disclaimer: this gist was drafted by Claude Code, which also generated the reduced reproducer.

Crash report

Re-importing a single-phase (legacy) C extension module routes through reload_singlephase_extension(). After the module is rebuilt it calls _modules_by_index_set() (Python/import.c:2010) to record it in the per-interpreter modules_by_index list. Under OOM that call fails (its internal PyList_New/PyList_Append raises MemoryError and returns -1). The error-cleanup branch then runs PyMapping_DelItem(modules, info->name) (import.c:2011) with the MemoryError still set. The delete from sys.modules (a dict) succeeds, so the debug check assert(_Py_CheckSlotResult(o, "__delitem__", res >= 0)) (Objects/abstract.c:280) sees success && PyErr_Occurred() and aborts the interpreter via _Py_FatalErrorFormat.

Reproducer

Minimal, stdlib-only (shrinkray-reduced from the vehicle, then cleaned; deterministic on the
ft_debug_asan build, re-verified 10/10). A dict __delitem__ slot succeeds with an
exception set under OOM. (This minimal repro is ft_debug_asan-specific; the build matrix
below is verified against the full vehicle_source.py, which also fatals on the jit debug
build via the readline single-phase-extension reload path the minimal form doesn't exercise.)

import faulthandler, pdb
faulthandler.enable()
from _testcapi import set_nomemory
for start in range(1000):
    set_nomemory(start)
    try:
        pdb.runcall()
    except BaseException:
        pass
print("done, no crash")

The full fuzzer vehicle is preserved as vehicle_source.py.

Backtrace

#8  _Py_CheckSlotResult          Objects/call.c:80      <- "Slot __delitem__ ... succeeded with an exception set"
#9  PyObject_DelItem             Objects/abstract.c:280 <- assert(_Py_CheckSlotResult(o,"__delitem__",res>=0))
#10 reload_singlephase_extension Python/import.c:2011   <- PyMapping_DelItem(modules, info->name) cleanup
#11 import_find_extension        Python/import.c:2043
#12 _imp_create_dynamic_impl     Python/import.c:5468

The faulting object is the live sys.modules dict and the delete succeeds
(res == 0); the pending object is a MemoryError (refcount 1) left set by the
failed _modules_by_index_set() at import.c:2010. This is a stale-exception /
missing-error-clear bug, not a NULL/freed pointer.

Root cause

Python/import.c, reload_singlephase_extension() (L2009-2014):

    Py_ssize_t index = _get_cached_module_index(cached);
    if (_modules_by_index_set(tstate->interp, index, mod) < 0) {   /* L2010: raises MemoryError under OOM */
        PyMapping_DelItem(modules, info->name);                    /* L2011: runs with MemoryError still set */
        Py_DECREF(mod);
        return NULL;
    }

_modules_by_index_set() (L577) can fail by raising MemoryError:

    if (MODULES_BY_INDEX(interp) == NULL) {
        MODULES_BY_INDEX(interp) = PyList_New(0);          /* alloc -> can fail */
        ...
    }
    while (PyList_GET_SIZE(MODULES_BY_INDEX(interp)) <= index) {
        if (PyList_Append(MODULES_BY_INDEX(interp), Py_None) < 0) {   /* alloc -> can fail */
            return -1;
        }
    }

When it returns -1 with MemoryError set, the cleanup calls
PyMapping_DelItem(modules, info->name) -> PyObject_DelItem ->
dict_ass_subscript (sys.modules is a dict). The key is present, so the slot
succeeds (res == 0). On a debug build, PyObject_DelItem
(Objects/abstract.c:280) wraps the call in
assert(_Py_CheckSlotResult(o, "__delitem__", res >= 0)), and
_Py_CheckSlotResult (Objects/call.c:86-90) treats "slot succeeded while an
exception is set" as a fatal interpreter invariant violation, calling
_Py_FatalErrorFormat. The defect is that the cleanup path performs an operation
that runs the dict __delitem__ slot without first preserving/clearing the
already-pending exception.

Suggested fix

Use the existing remove_module() cleanup helper instead of calling
PyMapping_DelItem(modules, info->name) directly in the
_modules_by_index_set() failure path:

    if (_modules_by_index_set(tstate->interp, index, mod) < 0) {
        remove_module(tstate, info->name);
        Py_DECREF(mod);
        return NULL;
    }
CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Output from running 'python -VV' on the command line:

No response

Linked PRs
  • gh-153798

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

从 Python/import.c 中 reload_singlephase_extension() 附近的第 2009-2014 行开始,然后阅读 remove_module() helper 和 _modules_by_index_set()。在 ft_debug_asan 构建上运行提供的最小 reproducer;当 OOM 清理不再触发 _Py_CheckSlotResult 或导致解释器 fatal 时,即表示完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
compilers
Issue 类型
缺陷
难度
3/5
预计耗时
1-2 天
活跃度
停滞
描述清晰度
描述清楚
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。