agronholm / agronholm/typeguard

Syntax error for custom type annotations

Open
#513 17 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
1.8k
Forks
145
Avg merge
8d 12h
Merged PRs (30d)
1

Description

### Things to check first

- [x] I have searched the existing issues and didn't find my bug already reported there

- [x] I have checked that my bug is still present in the latest release

### Typeguard version

4.4.1

### Python version

3.11

### What happened?

`typeguard.typechecked` fails with a syntax error for annotations that use item access with a string that is not `eval`-able. Basic example:

```python
import typeguard
annot = {"a b": int, "c d": str}

@typeguard.typechecked
def foo1(x: annot["a b"]) -> int:
return 10
```

yields
```
File "", line 1
a b
^
SyntaxError: invalid syntax
```

This problem occurs with custom annotation types like for example `jaxtyping` which uses annotations like `Float[np.ndarray, "H W C"]` to annotate shape information. The problem was previously raised in #353 for the special case of `Annotated`. The fix from [be4dd33](https://github.com/antonagestam/typeguard/commit/be4dd3350f15ec8b5713e40955dad3f4c9a68ad6) solves this for plain Annotated, but is quite brittle. For example this fails:

```python
import typing

MyAnnotated = typing.Annotated # simple alias

@typeguard.typechecked
def foo2(x: MyAnnotated[int, "a b"]) -> int:
return 10
```

The problem in both cases is that the instrumentation tries to evaluate "a b" as a forward reference which leads to a syntax error. I don't think this is the desired behaviour. Custom annotations are allowed by python and should not lead to errors. For reference, the `typing.get_type_hints()` function succeeds in both cases:

```python
typing.get_type_hints(foo1)
# {'x': int, 'return': int}

typing.get_type_hints(foo2)
# {'x': int, 'return': int}
```

I suggest to solve this problem by changing the instrumented code for a function to use `typing.get_type_hints()` rather than trying to manually catch all corner cases during the AST parsing.
Currently the instrumented code for a simple function looks like this:

```python
def bar(x: int, y: str) -> None:
...
return

# ---- instrumented ---
def bar(x: int, y: str) -> None:
from typeguard import TypeCheckMemo
from typeguard._functions import check_argument_types, check_return_type
memo = TypeCheckMemo(globals(), locals())
check_argument_types('bar', {'x': (x, int), 'y': (y, str)}, memo)
...
return check_return_type('bar', None, None, memo)
```

The problem could be fixed if instead trying to resolve the forward references during AST parsing, the `check_arguments_types` function simply called `typing.get_type_hints()`. So with a new `check_argument_types_v2`:
```python
def check_argument_types_v2(fun, args, memo):
name = fun.__name__
hints = typing.get_type_hints(fun)
annotated_arguments = {
k: (v, hints[k])
for k, v in args.items()
if k in hints
}
return check_argument_types(name, annotated_arguments, memo)
```

The instrumented code could become:
```python
def bar(x: int, y: str) -> None:
from typeguard import TypeCheckMemo
from typeguard._functions import check_argument_types, check_return_type
memo = TypeCheckMemo(globals(), locals())
check_argument_types_2(bar, {'x': x, 'y': y}, memo)
...
return check_return_type('bar', None, None, memo) # (should probably do the same here)
```

This should avoid all the above edge-cases and be pretty straightforward to implement.
I have a proof-of-concept working. Happy to submit it as a PR if that is wanted (though I am not very familiar with the AST-parsing code, and I don't feel confident in my edits).

### How can we reproduce the bug?

```python
import typeguard
annot = {"a b": int}

@typeguard.typechecked
def foo1(x: annot["a b"]) -> int:
return 10
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.