Instagram / Instagram/LibCST

RemoveImportsVisitor crashes when parsing Literal["r+"]

Open
#924 1 comment 1 reaction 0 assignees View on GitHub
bug codemod
Dominant language
Python
Stars
1.9k
Forks
229
PR merge metrics
No merged PRs in 30d

Description

https://peps.python.org/pep-0586/#shortening-unions-of-literals

The parser is trying to see if the string "r+" refers to some stringified-type and fails to parse it as an expression, but it should just take "r+" as a literal string.

This crashes in both the python and rust/native parser as of libcst 0.4.9

Repro:
```python
# foo.py
from typing import Literal
x: Literal["r+"] = "r+"
```

```python
# mods/mymod.py
import libcst as cst
from libcst.codemod import VisitorBasedCodemodCommand
from libcst.codemod.visitors import RemoveImportsVisitor

class CrashMod(VisitorBasedCodemodCommand):
DESCRIPTION: str = "Crash on literal"

def leave_AnnAssign(
self, original_node: cst.AnnAssign, updated_node: cst.AnnAssign,
) -> cst.AnnAssign:
RemoveImportsVisitor.remove_unused_import(self.context, "typing")
return updated_node
```

```yaml
# .libcst.codemod.yaml
generated_code_marker: '@generated'
formatter: ['black', '-']
blacklist_patterns: []
modules:
- 'mods'
repo_root: '.'

```

Output:
```
python -m libcst.tool codemod mymod.CrashMod foo.py
...
File "libcst/codemod/visitors/_gather_string_annotation_names.py", line 65, in visit_SimpleString
self.handle_any_string(node)
File "libcst/codemod/visitors/_gather_string_annotation_names.py", line 74, in handle_any_string
mod = cst.parse_module(value)
File "libcst/_parser/entrypoints.py", line 109, in parse_module
result = _parse(
File "libcst/_parser/entrypoints.py", line 56, in _parse
return _pure_python_parse(
File "libcst/_parser/entrypoints.py", line 89, in _pure_python_parse
result = parser.parse()
File "libcst/_parser/base_parser.py", line 110, in parse
self._add_token(token)
File "libcst/_parser/base_parser.py", line 186, in _add_token
raise ParserSyntaxError(
libcst._exceptions.ParserSyntaxError: Syntax Error @ 1:3.
Incomplete input. Unexpectedly encountered '\n'.

r+
^
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.