Debug abort when calling <generator>.throw(StopIteration) on a generator that hasn't started yet
还没有人认领这个 Issue。
- 主要语言
- Python
- 星标
- 77.2k
- 派生
- 35.9k
- PR 合并指标
- PR 指标待抓取
描述
Crash report
What happened?
In a debug build the following:
(_ for _ in ()).throw(StopIteration)
aborts with:
python3: Objects/genobject.c:318: PySendResult gen_send_ex2(PyGenObject *, PyObject *, PyObject **, int): Assertion `!PyErr_ExceptionMatches(PyExc_StopIteration)' failed.
Basically, calling generator.throw(StopIteration) on a generator that hasn't started yet triggers this.
In a non-debug build, the assert calls are not evaluated, but this very contrived example shows that there is a tortured violation of the PEP-479 fix that's possible (but unlikely to be seen in real code?):
Non-debug build script
from contextlib import contextmanager
@contextmanager
def transaction(level):
print(f"begin {level}")
try:
yield
print(f"commit {level}")
except:
print(f"rollback {level}")
raise
def bad_helper(start_first):
g = (_ for _ in (1, ))
if start_first:
next(g)
return g.throw(StopIteration)
def rows(start_first):
def next_row():
with transaction(2):
return bad_helper(start_first)
return iter(next_row, None)
def run(start_first):
print(f"Running with start_first={start_first}")
try:
with transaction(1):
for row in rows(start_first):
print(f"processed {row}")
except BaseException as e:
print(f"exception: {e!r}")
finally:
print("-=-=-=-=-=- STOP -=-=-=-=-=-")
run(False)
run(True)
Running this outputs:
Running with start_first=False
begin 1
begin 2
rollback 2
commit 1
-=-=-=-=-=- STOP -=-=-=-=-=-
Running with start_first=True
begin 1
begin 2
rollback 2
rollback 1
exception: RuntimeError('generator raised StopIteration')
-=-=-=-=-=- STOP -=-=-=-=-=-
You can see that if the generator is started, both levels of transaction roll back correctly and we get a RuntimeError. But if the generator is not started, then tx2 is rolled back, but tx1 is committed, and no exception raised.
What's happening
generator.throw()calls_gen_throw()
https://github.com/python/cpython/blob/ecdef1773006529b0fea6639d0effeecbb41679c/Objects/genobject.c#L756_gen_throwsets the exception, then callsgen_throw_current_exception...:
https://github.com/python/cpython/blob/ecdef1773006529b0fea6639d0effeecbb41679c/Objects/genobject.c#L609- ... which calls
gen_send_ex2 with exc=1:
https://github.com/python/cpython/blob/ecdef1773006529b0fea6639d0effeecbb41679c/Objects/genobject.c#L260 gen_send_ex2proceeds and around these lines:
https://github.com/python/cpython/blob/ecdef1773006529b0fea6639d0effeecbb41679c/Objects/genobject.c#L277-L281
re-enters the frame withthrowflag=1.- Looking at the bytecode for the frame:
Disassembly of <code object <genexpr> at 0xfffff7291360, file "eval", line 1>:
-- RESUME 4
1:16-1:21 LOAD_FAST 0 (.0)
1:16-1:21 GET_ITER 0
1:?-1:? RETURN_GENERATOR
1:?-1:? POP_TOP
1:0-1:0 L1: RESUME 0
1:16-1:21 L2: FOR_ITER 7 (to L3)
1:11-1:12 STORE_FAST_LOAD_FAST 17 (_, _)
1:5-1:6 YIELD_VALUE 0
1:5-1:6 RESUME 9
1:5-1:6 POP_TOP
1:5-1:6 JUMP_BACKWARD 9 (to L2)
1:16-1:21 L3: END_FOR
1:16-1:21 POP_ITER
1:16-1:21 LOAD_COMMON_CONSTANT 7 (None)
1:16-1:21 RETURN_VALUE
-- L4: CALL_INTRINSIC_1 3 (INTRINSIC_STOPITERATION_ERROR)
-- RERAISE 1
ExceptionTable:
L1 to L4 -> L4 [2] lasti
We see that the INTRINSIC_STOPITERATION_ERROR call at L4 resolves to stopiteration_error
which is where the Runtime Error is instantiated, and set:
https://github.com/python/cpython/blob/ecdef1773006529b0fea6639d0effeecbb41679c/Python/intrinsics.c#L172
- BUT if the iterator has not started, we haven't reached
L1yet (the initial resume wasn't called). so the Exception table does not cover those opcodes, and nostopiteration_errorwrapper is called.
The assertions are doing their fine job and catching the problem here, but I'm reasonably confident that the root issue is actually the opcode wrapping.
I'm not sure what the fix should be here, maybe it's as simple as somehow moving L1 up one opcode?
Introduced by
f4adb975061874566766f7a67206cb7b0439bc11 is the first bad commit
commit f4adb975061874566766f7a67206cb7b0439bc11
Author: Mark Shannon <mark@hotpy.org>
Date: Thu Nov 3 04:38:51 2022 -0700
GH-96793: Implement PEP 479 in bytecode. (GH-99006)
* Handle converting StopIteration to RuntimeError in bytecode.
* Add custom instruction for converting StopIteration into RuntimeError.
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS, Linux
Output from running 'python -VV' on the command line:
Python 3.14.5 (main, May 14 2026, 10:03:03) [GCC 15.2.1 20260209]
Linked PRs
- gh-154492
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
从 Objects/genobject.c 和 Python/intrinsics.c 中引用的路径开始,跟踪 _gen_throw()、gen_throw_current_exception()、gen_send_ex2() 和 stopiteration_error()。在调试构建中重现报告中的 generator.throw(StopIteration) 情况,然后添加或更新相关的回归测试覆盖,以便不再触发调试断言,并验证文档中所述的行为。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- python
- 领域
- compilers
- Issue 类型
- 缺陷
- 难度
- 5/5
- 预计耗时
- 一周以上
- 活跃度
- 停滞
- 描述清晰度
- 需要澄清
- 新手友好度
- 25/100