Use information from `@overload` to better model narrowing in implementation
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 20.6k
- Forks
- 3.3k
- PR merge metrics
- PR metrics pending
Description
Bug Report
There are potentially two issues that I have encountered:
- Consider a case where we use
@overloadand, for example, two arguments have narrower types than the signature of the non-@overload-decorated definition. If one of those is aLiteraltype and we branch on a comparison to the value of that literal type, the other argument isn't narrowed to the type(s) defined in the matching overloaded function(s).- This can be worked around by using
isinstance(), but it seems redundant. - It seems, according to Python's documentation that the current behaviour is sort of incorrect as it states:
The @overload-decorated definitions are for the benefit of the type checker only, since they will be overwritten by the non-@overload-decorated definition, while the latter is used at runtime but should be ignored by a type checker.
- But until either the literal comparison occurs or
isinstance()is checked on the other argument, it's not possible to narrow, but it should be after.
- This can be worked around by using
- The other issue is that, once we have called
isinstance()to narrow the type of a value, a subsequent nested function definition keeps the non-narrowed type when referring to the value.
See the reproducer below which should help clarify the above issues.
To Reproduce
See playground example.
from typing import Literal, overload
@overload
def f(name: Literal["integer"], value: int) -> int: ...
@overload
def f(name: Literal["string"], value: str) -> str: ...
def f(name: Literal["integer", "string"], value: int | str) -> int | str:
if name == "integer":
reveal_type(value)
assert isinstance(value, int)
reveal_type(value)
def do_integer() -> int:
reveal_type(value) # XXX: Expected `builtins.int`
return value # XXX: Expected no error.
reveal_type(value)
return do_integer()
if name == "string":
reveal_type(value)
assert isinstance(value, str)
reveal_type(value)
def do_string() -> str:
reveal_type(value) # XXX: Expected `builtins.str`
return value # XXX: Expected no error.
reveal_type(value)
return do_string()
Expected Behavior
Something like this mocked up output if the the type of value were narrowed based on the literal of name:
❯ mypy --strict bug.py
bug.py:11: note: Revealed type is "builtins.int"
bug.py:13: note: Revealed type is "builtins.int"
bug.py:16: note: Revealed type is "builtins.int"
bug.py:19: note: Revealed type is "builtins.int"
bug.py:24: note: Revealed type is "builtins.str"
bug.py:26: note: Revealed type is "builtins.str"
bug.py:29: note: Revealed type is "builtins.str"
bug.py:32: note: Revealed type is "builtins.str"
Success: no issues found in 1 source file
Something like this mocked up output if only the type inside the nested function respected the narrowing performed by the isinstance() calls:
❯ mypy --strict bug.py
bug.py:11: note: Revealed type is "Union[builtins.int, builtins.str]"
bug.py:13: note: Revealed type is "builtins.int"
bug.py:16: note: Revealed type is "builtins.int"
bug.py:19: note: Revealed type is "builtins.int"
bug.py:24: note: Revealed type is "Union[builtins.int, builtins.str]"
bug.py:26: note: Revealed type is "builtins.str"
bug.py:29: note: Revealed type is "builtins.str"
bug.py:32: note: Revealed type is "builtins.str"
Success: no issues found in 1 source file
Actual Behavior
❯ mypy --strict bug.py
bug.py:11: note: Revealed type is "Union[builtins.int, builtins.str]"
bug.py:13: note: Revealed type is "builtins.int"
bug.py:16: note: Revealed type is "Union[builtins.int, builtins.str]"
bug.py:17: error: Incompatible return value type (got "Union[int, str]", expected "int") [return-value]
bug.py:19: note: Revealed type is "builtins.int"
bug.py:24: note: Revealed type is "Union[builtins.int, builtins.str]"
bug.py:26: note: Revealed type is "builtins.str"
bug.py:29: note: Revealed type is "Union[builtins.int, builtins.str]"
bug.py:30: error: Incompatible return value type (got "Union[int, str]", expected "str") [return-value]
bug.py:32: note: Revealed type is "builtins.str"
Found 2 errors in 1 file (checked 1 source file)
Your Environment
- Mypy version used:
1.0.0 - Mypy command-line flags:
--strict - Mypy configuration options from
mypy.ini(and other config files): N/A - Python version used:
3.10.9
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked playground example or reproduce the shown bug.py command using mypy --strict, comparing the reveal_type output with the expected output. Investigate overload-based narrowing and narrowing captured by nested functions; done means the requested reveal_type results are produced and both incompatible return errors are gone.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100