Overlapping overloads in second-order functions cause returned value to be Any
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 20.6k
- Forks
- 3.3k
- PR merge metrics
- PR metrics pending
Description
Bug Report
Changing a parameter of a function from a determined type (e.g. int, str, etc.) to Any makes the return type of a second-order function applied on the initial function Any in some cases.
I originally found this issue using the pytest library (see in the wild section below).
To Reproduce
from typing import Any, Callable, overload, TypeVar
C = TypeVar("C", bound=Callable[..., object])
#
## the second-order function
#
@overload
def wrapper(arg: C) -> C:
...
@overload
def wrapper(arg: object) -> object:
...
def wrapper(arg: object) -> object:
return arg # we don't care
#
## the "working" case
#
def case_a(x: int) -> None:
pass
reveal_type(case_a) # Revealed type is "def (x: builtins.int)"
reveal_type(wrapper(case_a)) # Revealed type is "def (x: builtins.int)"
#
## the "issue" case: only the type of the `x` parameter has changed
#
def case_b(x: Any) -> None:
pass
reveal_type(case_b) # Revealed type is "def (x: Any)
reveal_type(wrapper(case_b)) # Revealed type is "Any"
Expected Behavior
reveal_type should give the same result for case_b and wrapper(case_b) (just like it did for case_a and wrapper(case_a)).
Actual Behavior
wrapper(case_b) is thought to have type Any
In the wild (pytest)
from typing import Any, Callable, TYPE_CHECKING, Union
import pytest
@pytest.mark.parametrize("obj", (42, "42"))
def test_bug0(obj: Union[int, str]) -> None:
assert len(str(obj)) < 20
@pytest.mark.parametrize("obj", (42, "42"))
def test_bug1(obj: Any) -> None:
assert len(str(obj)) < 20
@pytest.mark.parametrize("obj", (int, float))
def test_bug2(obj: Callable[[str], float]) -> None:
assert len(obj.__name__) < 20
@pytest.mark.parametrize("obj", (int, float))
def test_bug3(obj: Callable[..., Any]) -> None:
assert len(obj.__name__) < 20
if TYPE_CHECKING:
reveal_type(test_bug0) # Revealed type is "def (obj: Union[builtins.int, builtins.str])"
reveal_type(test_bug1) # Revealed type is "Any"
reveal_type(test_bug2) # Revealed type is "def (fct: def (builtins.str) -> builtins.float)"
reveal_type(test_bug3) # Revealed type is "Any"
We can see that test_bug0 and test_bug2 are well-typed, but test_bug1 and test_bug3 are typed as Any, which cause an error each (with --disallow-any-decorated):
$ mypy --disallow-any-decorated bug.py
bug.py:10: error: Function is untyped after decorator transformation
bug.py:18: error: Function is untyped after decorator transformation
bug.py:22: note: Revealed type is "def (obj: Union[builtins.int, builtins.str])"
bug.py:23: note: Revealed type is "Any"
bug.py:24: note: Revealed type is "def (fct: def (builtins.str) -> builtins.float)"
bug.py:25: note: Revealed type is "Any"
Found 2 errors in 1 file (checked 1 source file)
Here is the relevant parts of the source code of pytest for easy reference:
Markable = TypeVar("Markable", bound=Union[Callable[..., object], type])
class MarkDecorator:
# Type ignored because the overloads overlap with an incompatible
# return type. Not much we can do about that. Thankfully mypy picks
# the first match so it works out even if we break the rules.
@overload
def __call__(self, arg: Markable) -> Markable: # type: ignore[misc]
pass
@overload
def __call__(self, *args: object, **kwargs: object) -> "MarkDecorator":
pass
def __call__(self, *args: object, **kwargs: object):
pass # redacted
# pytest.mark.parametrize returns a MarkDecorator instance
Your Environment
- Mypy version used: 0.971
- Mypy command-line flags: none, except for the pytest case (see above)
- Mypy configuration options from
mypy.ini(and other config files): none - Python version used: 3.10
- Operating system and version: Arch Linux
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the provided wrapper/case_a/case_b reproducer and the pytest MarkDecorator.call example from src/_pytest/mark/structures.py. Trace overload resolution for the Any-parameter callable and compare the reveal_type results. Done means wrapper(case_b), test_bug1, and test_bug3 retain their callable signatures instead of becoming Any, while the existing cases remain unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100