Native parser crashes on pydoc_data.topics
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 229
- PR merge metrics
- No merged PRs in 30d
Description
I am trying to parse in the source files of the standard library. When I get to Lib/pydoc_data/topics.py I encounter issues.
The native parser of LibCST causes stack overflow on this file. It can be parsed correctly with the pure parser.
I am not sure how to debug it, but this file contains thousands of lines of ConcatenatedString nodes, so I would guess that causes the problem. If I use the pure parser, the code will error anyway on the cst.code expression with a RecursionError.
Tested on windows 10 with LibCST 1.0.1 on Python 3.11.4.
Code to reproduce:
```python
import sys, os, faulthandler
from pathlib import Path
import libcst
# os.environ["LIBCST_PARSER_TYPE"] = "pure"
content = (Path(sys.base_prefix) / "Lib/pydoc_data/topics.py").read_text("utf-8")
faulthandler.enable()
cst: libcst.Module = libcst.parse_module(content)
print(cst.code)
```
Result of the run:
```
Windows fatal exception: stack overflow
Current thread 0x000020f8 (most recent call first):
File "D:\projects\testing\libcst_error\.venv\Lib\site-packages\libcst\_parser\entrypoints.py", line 55 in _parse
File "D:\projects\testing\libcst_error\.venv\Lib\site-packages\libcst\_parser\entrypoints.py", line 109 in parse_module
File "D:\projects\testing\libcst_error\main.py", line 8 in
```
Contributor guide
Assessment
This issue has not been assessed yet.