PEP 515 Not Supported (Underscores in Numeric Literals)
- Dominant language
- Python
- Stars
- 14k
- Forks
- 904
- PR merge metrics
- No merged PRs in 30d
Description
Yapf 0.16.0 crashes on the following input (which is valid python code according to [PEP 515](https://www.python.org/dev/peps/pep-0515/)):
```python
million = 1000_000
```
raising this exception:
```
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/yapf/yapflib/pytree_utils.py", line 102, in ParseCodeToTree
tree = parser_driver.parse_string(code, debug=False)
File "/usr/lib/python3.6/lib2to3/pgen2/driver.py", line 106, in parse_string
return self.parse_tokens(tokens, debug)
File "/usr/lib/python3.6/lib2to3/pgen2/driver.py", line 71, in parse_tokens
if p.addtoken(type, value, (prefix, start)):
File "/usr/lib/python3.6/lib2to3/pgen2/parse.py", line 159, in addtoken
raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=1, value='_000', context=('', (1, 14))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/yapf", line 11, in
load_entry_point('yapf==0.16.0', 'console_scripts', 'yapf')()
File "/usr/lib/python3.6/site-packages/yapf/__init__.py", line 296, in run_main
sys.exit(main(sys.argv))
File "/usr/lib/python3.6/site-packages/yapf/__init__.py", line 188, in main
parallel=args.parallel)
File "/usr/lib/python3.6/site-packages/yapf/__init__.py", line 236, in FormatFiles
in_place, print_diff, verify)
File "/usr/lib/python3.6/site-packages/yapf/__init__.py", line 259, in _FormatFile
logger=logging.warning)
File "/usr/lib/python3.6/site-packages/yapf/yapflib/yapf_api.py", line 90, in FormatFile
verify=verify)
File "/usr/lib/python3.6/site-packages/yapf/yapflib/yapf_api.py", line 126, in FormatCode
tree = pytree_utils.ParseCodeToTree(unformatted_source)
File "/usr/lib/python3.6/site-packages/yapf/yapflib/pytree_utils.py", line 108, in ParseCodeToTree
tree = parser_driver.parse_string(code, debug=False)
File "/usr/lib/python3.6/lib2to3/pgen2/driver.py", line 106, in parse_string
return self.parse_tokens(tokens, debug)
File "/usr/lib/python3.6/lib2to3/pgen2/driver.py", line 71, in parse_tokens
if p.addtoken(type, value, (prefix, start)):
File "/usr/lib/python3.6/lib2to3/pgen2/parse.py", line 159, in addtoken
raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=1, value='_000', context=('', (1, 14))
```
But it works as expected on the following code:
```python
million = 1000000
```
Contributor guide
Assessment
This issue has not been assessed yet.