Memory errors during parsing may trigger a debug abort
Đang mở
Chưa có ai nhận issue này.
interpreter-core
topic-parser
type-crash
- Ngôn ngữ chính
- Python
- Star
- 77.2k
- Fork
- 35.9k
- Chỉ số merge pull request
- Chỉ số pull request đang chờ
Mô tả
Crash report
What happened?
import os
import sys
import _testcapi
source = b'''class E:
def __iter__(self):
yield 1
yield from ()
def f(a, b=0, *args, **kwargs): pass
f(*E(), **{&a": 3})'''
for counter in range(200, 1200):
print("Trying:", counter)
sys.stdout.flush()
_testcapi.set_nomemory(counter, counter + 1)
try:
_testcapi.run_compilestringflags(source, b"/missing.py", _testcapi.Py_file_input)
except Exception:
pass
_testcapi.remove_mem_hooks()
For me, I get (ymmv):
Trying: 200
Trying: 201
Trying: 202
...
Trying: 340
Trying: 341
python3: Objects/call.c:342: _PyObject_Call: Assertion `!_PyErr_Occurred(tstate)' failed.
This is related to these issues/prs:
A fix was put in for this type of issue already in _PyPegen_raise_error_known_location:
https://github.com/python/cpython/blob/e8158d1a02d6e5e974d4dbc0633b6a3ad960d7a3/Parser/pegen_errors.c#L276-L279
and _PyPegen_raise_error:
https://github.com/python/cpython/blob/e8158d1a02d6e5e974d4dbc0633b6a3ad960d7a3/Parser/pegen_errors.c#L185-L187
But in this case, PyErr_Occurred() == True but p->error_indicator=0, because the allocation failure happened in an area of the code that could backtrack, it seems like the error_indicator isn't set.
Two code patterns that hit both sites are:
class E:
def __iter__(self):
yield 1
yield from ()
def f(a, b=0, *args, **kwargs): pass
f(*E(), **{&a": 3})
and
class E:
def __iter__(self):
yield 1
yield from ()
def f(a, b=0, *args, **kwargs): pass
with f(*E())
pass
(The with here is the key)
So, to my mind, there are two questions:
- Is it valid to get to either of these places with PyErr_Occurred() but not p->error_indicator?
- If it is, then does the check just become checking
PyErr_Occurred()? - If not, then the places where the parser checks for memory error and returns should probably always set the parser error_indicator. I guess technically a mis-predicted parsing branch could cause a MemoryError, and then backtrack and find the correct branch without triggering another MemoryError?? but istm that if you hit an allocation failure on a backtrackable branch, the only valid thing to do is to bail out.
LLDB info
LLDB
(lldb) thread list
Process 2661 stopped
* thread #1: tid = 2661, 0x0000fffff692a848 libc.so.6`___lldb_unnamed_symbol_8a720 + 296, name = 'fuzz_python', stop reason = SIGABRT: sent by tkill system call (sender pid=2661, uid=0)
(lldb) bt all
* thread #1, name = 'fuzz_python', stop reason = SIGABRT: sent by tkill system call (sender pid=2661, uid=0)
* frame #0: 0x0000fffff692a848 libc.so.6`___lldb_unnamed_symbol_8a720 + 296
frame #1: 0x0000fffff68d7af8 libc.so.6`raise + 24
frame #2: 0x0000fffff68c2630 libc.so.6`abort + 40
frame #3: 0x0000fffff691dd84 libc.so.6`___lldb_unnamed_symbol_7dbe0 + 420
frame #4: 0x0000fffff68d13ac libc.so.6`__assert_fail + 204
frame #5: 0x0000aaaaaaee0834 fuzz_python`_PyObject_Call(tstate=0x0000aaaaab776358, callable=0x0000aaaaab6f4100, args=0x0000fffff4eb8f50, kwargs=0x0000000000000000) at call.c:342:5
frame #6: 0x0000aaaaaad35d70 fuzz_python`PyObject_Call(callable=0x0000aaaaab6f4100, args=0x0000fffff4eb8f50, kwargs=0x0000000000000000) at call.c:373:12 [inlined]
frame #7: 0x0000aaaaaad35d5c fuzz_python`PyErr_SetFromErrnoWithFilenameObjects(exc=0x0000aaaaab6f4100, filenameObject=0x0000fffff507eb60, filenameObject2=0x0000000000000000) at errors.c:900:13
frame #8: 0x0000aaaaab0f5a7c fuzz_python`PyErr_SetFromErrnoWithFilenameObject(exc=<unavailable>, filenameObject=0x0000fffff507eb60) at errors.c:811:12 [inlined]
frame #9: 0x0000aaaaab0f5a78 fuzz_python`Py_fopen(path=0x0000fffff507eb60, mode="rb") at fileutils.c:1817:9
frame #10: 0x0000aaaaaad3ce30 fuzz_python`_PyErr_ProgramDecodedTextObject(filename=0x0000fffff507eb60, lineno=7, encoding=0x0000000000000000) at errors.c:2064:16
frame #11: 0x0000aaaaab23e734 fuzz_python`_PyPegen_raise_error_known_location(p=0x0000fffff516a740, errtype=0x0000aaaaab6f6bd8, lineno=7, col_offset=12, end_lineno=7, end_col_offset=13, errmsg="Invalid star expression", va=<unavailable>) at pegen_errors.c:302:22
frame #12: 0x0000aaaaab23e360 fuzz_python`_PyPegen_raise_error(p=0x0000fffff516a740, errtype=0x0000aaaaab6f6bd8, use_mark=0, errmsg="Invalid star expression") at pegen_errors.c:221:5
frame #13: 0x0000aaaaab2fad34 fuzz_python`invalid_starred_expression_rule(p=0x0000fffff516a740) at parser.c:27408:20
frame #14: 0x0000aaaaab2fa614 fuzz_python`starred_expression_rule(p=0x0000fffff516a740) at parser.c:18983:47
frame #15: 0x0000aaaaab2d59a8 fuzz_python`_tmp_89_rule(p=0x0000fffff516a740) at parser.c:34278:39
frame #16: 0x0000aaaaab2d58a0 fuzz_python`genexp_rule(p=0x0000fffff516a740) at parser.c:18464:18
frame #17: 0x0000aaaaab2f3e68 fuzz_python`t_primary_raw(p=0x0000fffff516a740) at parser.c:20208:18
frame #18: 0x0000aaaaab2f3400 fuzz_python`t_primary_rule(p=0x0000fffff516a740) at parser.c:20077:22
frame #19: 0x0000aaaaab34a674 fuzz_python`single_subscript_attribute_target_rule(p=0x0000fffff516a740) at parser.c:19968:18
frame #20: 0x0000aaaaab32773c fuzz_python`_tmp_11_rule(p=0x0000fffff516a740) at parser.c:29309:54
frame #21: 0x0000aaaaab2b9878 fuzz_python`assignment_rule(p=0x0000fffff516a740) at parser.c:2196:18
frame #22: 0x0000aaaaab2b97cc fuzz_python`simple_stmt_rule(p=0x0000fffff516a740) at parser.c:1620:31
frame #23: 0x0000aaaaab2ae8c0 fuzz_python`simple_stmts_rule(p=0x0000fffff516a740) at parser.c:1515:18
frame #24: 0x0000aaaaab2aafc0 fuzz_python`statement_rule(p=0x0000fffff516a740) at parser.c:1297:34
frame #25: 0x0000aaaaab2aad28 fuzz_python`_loop1_2_rule(p=0x0000fffff516a740) at parser.c:28730:30
frame #26: 0x0000aaaaab2aab60 fuzz_python`statements_rule(p=0x0000fffff516a740) at parser.c:1230:18
frame #27: 0x0000aaaaab2a7578 fuzz_python`file_rule(p=0x0000fffff516a740) at parser.c:1032:18
frame #28: 0x0000aaaaab2a7510 fuzz_python`_PyPegen_parse(p=0x0000fffff516a740) at parser.c:39618:18
frame #29: 0x0000aaaaab22d3cc fuzz_python`_PyPegen_run_parser(p=0x0000fffff516a740) at pegen.c:1019:9
frame #30: 0x0000aaaaab22f308 fuzz_python`_PyPegen_run_parser_from_string(str="class E:\n def __iter__(self):\n yield 1\n yield from ()\ndef f(a, b=0, *urnc, **d):\n return a, b, c, d\nf(*E(), **{&a\": 3})", start_rule=257, filename_ob=0x0000fffff507eb60, flags=0x0000000000000000, arena=0x0000fffff51772e0, module=<unavailable>) at pegen.c:1145:14
frame #31: 0x0000aaaaab0f1c14 fuzz_python`_PyParser_ASTFromString(str="class E:\n def __iter__(self):\n yield 1\n yield from ()\ndef f(a, b=0, *urnc, **d):\n return a, b, c, d\nf(*E(), **{&a\": 3})", filename=0x0000fffff507eb60, mode=257, flags=0x0000000000000000, arena=0x0000fffff51772e0, module=0x0000000000000000) at peg_api.c:14:21 [inlined]
frame #32: 0x0000aaaaab0f1bd8 fuzz_python`_Py_CompileString(str="class E:\n def __iter__(self):\n yield 1\n yield from ()\ndef f(a, b=0, *urnc, **d):\n return a, b, c, d\nf(*E(), **{&a\": 3})", filename=0x0000fffff507eb60, start=257, flags=0x0000000000000000, optimize=-1, module=0x0000000000000000) at pythonrun.c:1578:11
frame #33: 0x0000aaaaab0f1ef0 fuzz_python`Py_CompileStringObject(str="class E:\n def __iter__(self):\n yield 1\n yield from ()\ndef f(a, b=0, *urnc, **d):\n return a, b, c, d\nf(*E(), **{&a\": 3})", filename=0x0000fffff507eb60, start=257, flags=0x0000000000000000, optimize=-1) at pythonrun.c:1561:12 [inlined]
frame #34: 0x0000aaaaab0f1ed4 fuzz_python`Py_CompileStringExFlags(str="class E:\n def __iter__(self):\n yield 1\n yield from ()\ndef f(a, b=0, *urnc, **d):\n return a, b, c, d\nf(*E(), **{&a\": 3})", filename_str="<fuzz>", start=257, flags=0x0000000000000000, optimize=-1) at pythonrun.c:1608:10
frame #35: 0x0000aaaaaad15054 fuzz_python`main(argc=<unavailable>, argv=<unavailable>) at fuzz_python.c:576:30
frame #36: 0x0000fffff68c2d94 libc.so.6`___lldb_unnamed_symbol_22d20 + 116
frame #37: 0x0000fffff68c2ef8 libc.so.6`__libc_start_main + 152
frame #38: 0x0000aaaaaad06370 fuzz_python`_start + 48
(lldb) register read
General Purpose Registers:
x0 = 0x0000000000000000
x1 = 0x0000000000000a65
x2 = 0x0000000000000006
x3 = 0x0000000000000022
x4 = 0xffffffffffffffff
x5 = 0x0000000000000000
x6 = 0x0000aaaaaab9e165
x7 = 0x0000fffff69f9750
x8 = 0x0000000000000083
x9 = 0x5210160bf182cd92
x10 = 0x0000000000000021
x11 = 0x0000fffff6a66618
x12 = 0x0000000000000480
x13 = 0x0000fffff7f03740
x14 = 0x0000fffff51036f0
x15 = 0x0000000000000001
x16 = 0x0000000000000001
x17 = 0x0000fffff693a36c libc.so.6`__libc_free
x18 = 0x0000000000000002
x19 = 0x0000000000000a65
x20 = 0x0000fffff7f03020
x21 = 0x000000000000000b
x22 = 0x0000000000000006
x23 = 0x0000000000000000
x24 = 0x000000000000009d
x25 = 0x0000ffffffff7a38
x26 = 0x0000fffff69f9740
x27 = 0x0000000000000002
x28 = 0x0000fffff51a0000
fp = 0x0000ffffffff7900
lr = 0x0000fffff692a834 libc.so.6`___lldb_unnamed_symbol_8a720 + 276
sp = 0x0000ffffffff78f0
pc = 0x0000fffff692a848 libc.so.6`___lldb_unnamed_symbol_8a720 + 296
cpsr = 0x60001000
(lldb) memory region $sp
[0x0000fffffffdd000-0x0001000000000000) rw-
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
Python 3.16.0a0 (heads/main-dirty:20e6c2f, Aug 19 2026, 15:22:30) [Clang 22.1.8 ]
Hướng dẫn đóng góp
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Hướng nghiên cứu
Bắt đầu bằng cách tái hiện lỗi với _testcapi memory-failure loop được cung cấp, sau đó đọc Parser/pegen_errors.c và assertion trong Objects/call.c. Theo dõi cách parser xử lý error_indicator đối với hai mẫu mã được cung cấp và so sánh với các đường dẫn lỗi được liên kết. Được xem là hoàn tất khi reproducer gây ra hoặc báo cáo lỗi cấp phát mà không kích hoạt debug abort.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- c, python
- Lĩnh vực
- compilers
- Loại issue
- Lỗi
- Độ khó
- 4/5
- Thời gian dự kiến
- 3-5 ngày
- Mức độ hoạt động
- Ít trao đổi
- Độ rõ ràng
- Khá rõ ràng
- Mức phù hợp với người mới
- 45/100