python / python/mypy

Decorated __init__ confuses callback type if decorator is defined with type alias or if decorator is imported using relative import

Open
#11,293 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug topic-type-alias
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

This one's just weird. I'm not sure if this is a dup of #1927, but I figured I'd raise it anyway.

# test_case.py
from typing import Callable, Iterable, TypeVar, Union

_T = TypeVar("_T")

def _identity(__: _T) -> _T:
    return __

# decr: Callable[[_T], _T] = _identity  # <-- this approach works
DecoratorT = Callable[[_T], _T]
decr: DecoratorT = _identity  # <-- this causes the false positives below

class Result:
    @decr
    def __init__(self):
        pass

res = Result()  # ; reveal_type(res)  # -> Any (should be test_case.Result)

def some_results(op: Callable[[], Union[Result, Iterable[Result]]]) -> Iterable[Result]:
    result_or_results = op()
    results: Iterable[Result]
    if isinstance(result_or_results, Result):
        results = (result_or_results,)  # ; reveal_type(result_or_results)  # -> (should be test_case.Result)
    else:
        results = result_or_results  # ; reveal_type(result_or_results)  # -> Union[test_case.Result, typing.Iterable[test_case.Result]] (should be typing.Iterable[test_case.Result])
    return results
% python --version ; mypy --version
Python 3.9.7
mypy 0.910
% mypy --config-file=pyproject.toml test_case.py
test_case.py:26: error: Incompatible types in assignment (expression has type "Union[Result, Iterable[Result]]", variable has type "Iterable[Result]")
Found 1 error in 1 file (checked 1 source file)

Note that if DecoratorT is defined as follows, things work again:

# …
class DecoratorT(Protocol):
    def __call__(self, __: _T) -> _T:
        ...
decr: DecoratorT = _identity  # <-- all good
# …

But if we move the decorator into its own module and use a relative import, it breaks no matter how we define the decorator type:

# test_case.py
from typing import Callable, Iterable, TypeVar, Union
# from test_case_import import decr  # <-- this works
from .test_case_import import decr  # <-- this causes the false positives below 

class Result:
    @decr
    def __init__(self):
        pass

res = Result()  # ; reveal_type(res)  # -> Any (should be test_case.Result)

def some_results(op: Callable[[], Union[Result, Iterable[Result]]]) -> Iterable[Result]:
    result_or_results = op()
    results: Iterable[Result]
    if isinstance(result_or_results, Result):
        results = (result_or_results,)  # ; reveal_type(result_or_results)  # -> (should be test_case.Result)
    else:
        results = result_or_results  # ; reveal_type(result_or_results)  # -> Union[test_case.Result, typing.Iterable[test_case.Result]] (should be typing.Iterable[test_case.Result])
    return results
# test_case_import.py
from typing import Callable, TypeVar
_T = TypeVar("_T")

def _identity(__: _T) -> _T:
    return __

decr: Callable[[_T], _T] = _identity  # <-- doesn't matter how this is defined
% mypy --config-file=pyproject.toml test_case.py test_case_import.py
test_case.py:19: error: Incompatible types in assignment (expression has type "Union[Result, Iterable[Result]]", variable has type "Iterable[Result]")
Found 1 error in 1 file (checked 2 source files)

Using the Protocol approach does not salvage the second (relative import) scenario.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported diagnostics with mypy on test_case.py and test_case_import.py, comparing the local and relative-import decorator cases. Trace how mypy infers the decorated init and imported decorator types. Done means the examples infer Result and Iterable[Result] as annotated without the false-positive assignment error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.