agronholm / agronholm/typeguard
Syntax error for custom type annotations
- Dominant language
- Python
- Stars
- 1.8k
- Forks
- 145
- Avg merge
- 8d 12h
- Merged PRs (30d)
- 1
Description
### Things to check first
- [x] I have searched the existing issues and didn't find my bug already reported there
- [x] I have checked that my bug is still present in the latest release
### Typeguard version
4.4.1
### Python version
3.11
### What happened?
`typeguard.typechecked` fails with a syntax error for annotations that use item access with a string that is not `eval`-able. Basic example:
```python
import typeguard
annot = {"a b": int, "c d": str}
@typeguard.typechecked
def foo1(x: annot["a b"]) -> int:
return 10
```
yields
```
File "", line 1
a b
^
SyntaxError: invalid syntax
```
This problem occurs with custom annotation types like for example `jaxtyping` which uses annotations like `Float[np.ndarray, "H W C"]` to annotate shape information. The problem was previously raised in #353 for the special case of `Annotated`. The fix from [be4dd33](https://github.com/antonagestam/typeguard/commit/be4dd3350f15ec8b5713e40955dad3f4c9a68ad6) solves this for plain Annotated, but is quite brittle. For example this fails:
```python
import typing
MyAnnotated = typing.Annotated # simple alias
@typeguard.typechecked
def foo2(x: MyAnnotated[int, "a b"]) -> int:
return 10
```
The problem in both cases is that the instrumentation tries to evaluate "a b" as a forward reference which leads to a syntax error. I don't think this is the desired behaviour. Custom annotations are allowed by python and should not lead to errors. For reference, the `typing.get_type_hints()` function succeeds in both cases:
```python
typing.get_type_hints(foo1)
# {'x': int, 'return': int}
typing.get_type_hints(foo2)
# {'x': int, 'return': int}
```
I suggest to solve this problem by changing the instrumented code for a function to use `typing.get_type_hints()` rather than trying to manually catch all corner cases during the AST parsing.
Currently the instrumented code for a simple function looks like this:
```python
def bar(x: int, y: str) -> None:
...
return
# ---- instrumented ---
def bar(x: int, y: str) -> None:
from typeguard import TypeCheckMemo
from typeguard._functions import check_argument_types, check_return_type
memo = TypeCheckMemo(globals(), locals())
check_argument_types('bar', {'x': (x, int), 'y': (y, str)}, memo)
...
return check_return_type('bar', None, None, memo)
```
The problem could be fixed if instead trying to resolve the forward references during AST parsing, the `check_arguments_types` function simply called `typing.get_type_hints()`. So with a new `check_argument_types_v2`:
```python
def check_argument_types_v2(fun, args, memo):
name = fun.__name__
hints = typing.get_type_hints(fun)
annotated_arguments = {
k: (v, hints[k])
for k, v in args.items()
if k in hints
}
return check_argument_types(name, annotated_arguments, memo)
```
The instrumented code could become:
```python
def bar(x: int, y: str) -> None:
from typeguard import TypeCheckMemo
from typeguard._functions import check_argument_types, check_return_type
memo = TypeCheckMemo(globals(), locals())
check_argument_types_2(bar, {'x': x, 'y': y}, memo)
...
return check_return_type('bar', None, None, memo) # (should probably do the same here)
```
This should avoid all the above edge-cases and be pretty straightforward to implement.
I have a proof-of-concept working. Happy to submit it as a PR if that is wanted (though I am not very familiar with the AST-parsing code, and I don't feel confident in my edits).
### How can we reproduce the bug?
```python
import typeguard
annot = {"a b": int}
@typeguard.typechecked
def foo1(x: annot["a b"]) -> int:
return 10
```
Contributor guide
Assessment
This issue has not been assessed yet.