google / google/yapf

Improve error reporting when `yapf` encounters unsupported Python syntax

Open
#1,194 0 comments 3 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
14k
Forks
904
PR merge metrics
No merged PRs in 30d

Description

I've encountered a problem with `yapf==0.32` when processing code with multiple context managers. Here is an example.
```python
# context.py

from contextlib import contextmanager

@contextmanager
def manager1(x):
yield x + 1
print("manager1 yielded")

@contextmanager
def manager2(y):
yield y ** 2
print("manager2 yielded")

def main():
with (
manager1(1) as first,
manager2(2) as second,
):
print(f"{first=}, {second=}")

if __name__ == "__main__":
main()
```
See the `main` function. The tool doesn't like context managers separated with commas.

However, my question is **not** about unsupported language features, but rather about _error reporting_. This is what I see as the output. (Truncated to make it more readable.)
```bash
> yapf context.py
Traceback (most recent call last):
...
raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=1, value='as', context=(' ', (16, 20))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
...
raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=1, value='as', context=(' ', (16, 20))

During handling of the above exception, another exception occurred:
...
return '{}:{}:{}: {}'.format(e.args[1][0], e.args[1][1], e.args[1][2], e.msg)
IndexError: tuple index out of range

During handling of the above exception, another exception occurred:
...

return '{}:{}:{}: {}'.format(e.args[1][0], e.args[1][1], e.args[1][2], e.msg)
IndexError: tuple index out of range
```
Unfortunately, none of these lines tell you where exactly this parsing error happened.

I've stumbled upon this kind of error multiple times already, with different language features being unsupported by a version of `yapf` used at that time. Updating the version usually helps, but it would be nice to see files that break the tool. We use `yapf` as a pre-commit hook, and it might be difficult to find what exactly fails when there are multiple files changed in a pull request. Seeing the origin of the error would help to roll back to supported syntax until the tool is updated to a newer version.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.