YAPF bails on utf-8 encoded files.
Open
- Dominant language
- Python
- Stars
- 14k
- Forks
- 904
- PR merge metrics
- No merged PRs in 30d
Description
e.g. https://github.com/Ezhil-Language-Foundation/open-tamil/blob/master/solthiruthi/peyarchorkal.py file will not be reformatted correctly.
sys.exit(main(sys.argv))
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/yapf/__init__.py", line 135, in main
verbose=args.verbose)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/yapf/__init__.py", line 204, in FormatFiles
in_place, print_diff, verify, quiet, verbose)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/yapf/__init__.py", line 233, in _FormatFile
logger=logging.warning)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/yapf/yapflib/yapf_api.py", line 100, in FormatFile
verify=verify)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/yapf/yapflib/yapf_api.py", line 147, in FormatCode
tree = pytree_utils.ParseCodeToTree(unformatted_source)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/yapf/yapflib/pytree_utils.py", line 121, in ParseCodeToTree
tree = parser_driver.parse_string(code, debug=False)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib2to3/pgen2/driver.py", line 106, in parse_string
return self.parse_tokens(tokens, debug)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib2to3/pgen2/driver.py", line 71, in parse_tokens
if p.addtoken(type, value, (prefix, start)):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib2to3/pgen2/parse.py", line 116, in addtoken
ilabel = self.classify(type, value, context)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib2to3/pgen2/parse.py", line 172, in classify
raise ParseError("bad token", type, value, context)
lib2to3.pgen2.parse.ParseError: bad token: type=58, value='ப', context=('', (23, 22))
Contributor guide
Assessment
This issue has not been assessed yet.