python / python/cpython

stale `MemoryError` trips `_Py_CheckSlotResult` in `reload_singlephase_extension`

オープン
#153,797 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

interpreter-core type-crash
主要言語
Python
スター
77.2k
フォーク
35.9k
PR マージ指標
PR 指標を取得中

説明

Crash report

What happened?

This is sub-issue of https://github.com/python/cpython/issues/151763
When _modules_by_index_set() fails under OOM, the cleanup PyMapping_DelItem(modules, name) runs the dict __delitem__ slot with the MemoryError still set; the delete succeeds with an exception pending, so the debug _Py_CheckSlotResult assert fatals the interpreter.

AI Disclaimer: this gist was drafted by Claude Code, which also generated the reduced reproducer.

Crash report

Re-importing a single-phase (legacy) C extension module routes through reload_singlephase_extension(). After the module is rebuilt it calls _modules_by_index_set() (Python/import.c:2010) to record it in the per-interpreter modules_by_index list. Under OOM that call fails (its internal PyList_New/PyList_Append raises MemoryError and returns -1). The error-cleanup branch then runs PyMapping_DelItem(modules, info->name) (import.c:2011) with the MemoryError still set. The delete from sys.modules (a dict) succeeds, so the debug check assert(_Py_CheckSlotResult(o, "__delitem__", res >= 0)) (Objects/abstract.c:280) sees success && PyErr_Occurred() and aborts the interpreter via _Py_FatalErrorFormat.

Reproducer

Minimal, stdlib-only (shrinkray-reduced from the vehicle, then cleaned; deterministic on the
ft_debug_asan build, re-verified 10/10). A dict __delitem__ slot succeeds with an
exception set under OOM. (This minimal repro is ft_debug_asan-specific; the build matrix
below is verified against the full vehicle_source.py, which also fatals on the jit debug
build via the readline single-phase-extension reload path the minimal form doesn't exercise.)

import faulthandler, pdb
faulthandler.enable()
from _testcapi import set_nomemory
for start in range(1000):
    set_nomemory(start)
    try:
        pdb.runcall()
    except BaseException:
        pass
print("done, no crash")

The full fuzzer vehicle is preserved as vehicle_source.py.

Backtrace

#8  _Py_CheckSlotResult          Objects/call.c:80      <- "Slot __delitem__ ... succeeded with an exception set"
#9  PyObject_DelItem             Objects/abstract.c:280 <- assert(_Py_CheckSlotResult(o,"__delitem__",res>=0))
#10 reload_singlephase_extension Python/import.c:2011   <- PyMapping_DelItem(modules, info->name) cleanup
#11 import_find_extension        Python/import.c:2043
#12 _imp_create_dynamic_impl     Python/import.c:5468

The faulting object is the live sys.modules dict and the delete succeeds
(res == 0); the pending object is a MemoryError (refcount 1) left set by the
failed _modules_by_index_set() at import.c:2010. This is a stale-exception /
missing-error-clear bug, not a NULL/freed pointer.

Root cause

Python/import.c, reload_singlephase_extension() (L2009-2014):

    Py_ssize_t index = _get_cached_module_index(cached);
    if (_modules_by_index_set(tstate->interp, index, mod) < 0) {   /* L2010: raises MemoryError under OOM */
        PyMapping_DelItem(modules, info->name);                    /* L2011: runs with MemoryError still set */
        Py_DECREF(mod);
        return NULL;
    }

_modules_by_index_set() (L577) can fail by raising MemoryError:

    if (MODULES_BY_INDEX(interp) == NULL) {
        MODULES_BY_INDEX(interp) = PyList_New(0);          /* alloc -> can fail */
        ...
    }
    while (PyList_GET_SIZE(MODULES_BY_INDEX(interp)) <= index) {
        if (PyList_Append(MODULES_BY_INDEX(interp), Py_None) < 0) {   /* alloc -> can fail */
            return -1;
        }
    }

When it returns -1 with MemoryError set, the cleanup calls
PyMapping_DelItem(modules, info->name) -> PyObject_DelItem ->
dict_ass_subscript (sys.modules is a dict). The key is present, so the slot
succeeds (res == 0). On a debug build, PyObject_DelItem
(Objects/abstract.c:280) wraps the call in
assert(_Py_CheckSlotResult(o, "__delitem__", res >= 0)), and
_Py_CheckSlotResult (Objects/call.c:86-90) treats "slot succeeded while an
exception is set" as a fatal interpreter invariant violation, calling
_Py_FatalErrorFormat. The defect is that the cleanup path performs an operation
that runs the dict __delitem__ slot without first preserving/clearing the
already-pending exception.

Suggested fix

Use the existing remove_module() cleanup helper instead of calling
PyMapping_DelItem(modules, info->name) directly in the
_modules_by_index_set() failure path:

    if (_modules_by_index_set(tstate->interp, index, mod) < 0) {
        remove_module(tstate, info->name);
        Py_DECREF(mod);
        return NULL;
    }
CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Output from running 'python -VV' on the command line:

No response

Linked PRs
  • gh-153798

コントリビューションガイド

コントリビューションガイドを開く

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

調査の方向性

Python/import.c の reload_singlephase_extension() 付近(2009~2014 行)から始め、次に remove_module() ヘルパーと _modules_by_index_set() を読んでください。提供されている最小 reproducer を ft_debug_asan ビルドで実行してください。OOM クリーンアップによって _Py_CheckSlotResult がトリガーされなくなり、インタープリターが fatal にならなくなれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
python
領域
compilers
issue の種類
バグ
難易度
3/5
見積もり時間
1〜2日
活発さ
停滞
明瞭さ
明確に書かれている
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。