Unicode line separator und newline break parsing
- Dominant language
- Python
- Stars
- 14k
- Forks
- 904
- PR merge metrics
- No merged PRs in 30d
Description
There are various line breaking characters, and these can be included in strings. Python can happily parse string literals that contain them. But YAPF fails to parse them, crashing with an EOS in a string literal.
Example tokens which crash YAPF are these:
- [U+0013](https://www.compart.com/en/unicode/U+0013): Device Control 3
- [U+001c](https://www.compart.com/en/unicode/U+001C): File Separator
- [U+0085](https://www.compart.com/de/unicode/U+0085): Next Line
- [U+2028](https://www.compart.com/en/unicode/U+2028): Line Separator
We might be able to escape these unicode literals with an escape code, but we'd prefer not to do it. Is there a way to make YAPF successfully parse the same files which the Python interpreter can also parse?
Contributor guide
Assessment
This issue has not been assessed yet.