google / google/yapf

YAPF bails on utf-8 encoded files.

Open
#891 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
14k
Forks
904
PR merge metrics
No merged PRs in 30d

Description

e.g. https://github.com/Ezhil-Language-Foundation/open-tamil/blob/master/solthiruthi/peyarchorkal.py file will not be reformatted correctly.

    sys.exit(main(sys.argv))

File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/yapf/__init__.py", line 135, in main
verbose=args.verbose)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/yapf/__init__.py", line 204, in FormatFiles
in_place, print_diff, verify, quiet, verbose)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/yapf/__init__.py", line 233, in _FormatFile
logger=logging.warning)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/yapf/yapflib/yapf_api.py", line 100, in FormatFile
verify=verify)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/yapf/yapflib/yapf_api.py", line 147, in FormatCode
tree = pytree_utils.ParseCodeToTree(unformatted_source)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/yapf/yapflib/pytree_utils.py", line 121, in ParseCodeToTree
tree = parser_driver.parse_string(code, debug=False)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib2to3/pgen2/driver.py", line 106, in parse_string
return self.parse_tokens(tokens, debug)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib2to3/pgen2/driver.py", line 71, in parse_tokens
if p.addtoken(type, value, (prefix, start)):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib2to3/pgen2/parse.py", line 116, in addtoken
ilabel = self.classify(type, value, context)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib2to3/pgen2/parse.py", line 172, in classify
raise ParseError("bad token", type, value, context)
lib2to3.pgen2.parse.ParseError: bad token: type=58, value='ப', context=('', (23, 22))

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.