python / python/mypy

Incorrect type inference with generic decorator type

Open
#11,369 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

I'm trying to define a generic type alias for a return-type-changing decorator:

T = TypeVar("T")
R = TypeVar("R")
Decorator = Callable[[Callable[..., T]], Callable[..., R]]

So for example, if a decorator wraps a return value in a list, its type could be Decorator[T, List[T]], and it's much clearer than nested Callables.

However, this doesn't work. Here's an example (also see on mypy-play):

from typing import *

T = TypeVar("T")
R = TypeVar("R")
Decorator = Callable[[Callable[..., T]], Callable[..., R]]

# def to_list() -> Decorator[T, List[T]]:
def to_list() -> Callable[[Callable[..., T]], Callable[..., List[T]]]:
    def decorator(func):
        def wrapped(*args, **kwargs):
            return [func(*args, **kwargs)]
        return wrapped
    return decorator
reveal_type(to_list)
reveal_type(to_list())

@to_list()
def foo() -> int:
    return 1

assert foo() == [1]
reveal_type(foo())


def to_list2() -> Decorator[T, List[T]]: ...
reveal_type(to_list2)
reveal_type(to_list2())

@to_list2()
def foo2() -> int:
    return 1
    
assert foo() == [1]
reveal_type(foo2())

Expected Behavior

All things type-check and foo has the same type as foo2.

Actual Behavior

main.py:14: note: Revealed type is "def () -> def [T] (def (*Any, **Any) -> T`-1) -> def (*Any, **Any) -> builtins.list[T`-1]"
main.py:15: note: Revealed type is "def [T] (def (*Any, **Any) -> T`-1) -> def (*Any, **Any) -> builtins.list[T`-1]"
main.py:22: note: Revealed type is "builtins.list[builtins.int*]"
main.py:26: note: Revealed type is "def [T] () -> def (def (*Any, **Any) -> T`-1) -> def (*Any, **Any) -> builtins.list[T`-1]"
main.py:27: note: Revealed type is "def (def (*Any, **Any) -> <nothing>) -> def (*Any, **Any) -> builtins.list[<nothing>]"
main.py:29: error: Argument 1 has incompatible type "Callable[[], int]"; expected "Callable[..., <nothing>]"
main.py:34: note: Revealed type is "builtins.list[<nothing>]"
Found 1 error in 1 file (checked 1 source file)

Mypy inferred foo correctly but not foo2. Somehow <nothing> got in where there should be TypeVars. The signatures for to_list and to_list2 are also different and I don't really understand why.

I'm guessing it's because I'm substituting two different TypeVars (T and R) with type expressions that reference the same TypeVar T? But I can't really find a formal definition of how TypeVars work, so I'm not sure if this is supported or not.

Your Environment

  • Mypy version used: latest
  • Mypy command-line flags: default
  • Python version used: 3.10
  • Operating system and version: mypy-play

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the supplied mypy-play reproduction with the reported Python 3.10 environment and compare the revealed types and error with the expected behavior. Trace the generic decorator type inference involved in Decorator, to_list, and to_list2; done means both forms type-check and foo and foo2 receive equivalent list-returning types.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.