python / python/mypy

Type narrowing not found for combination of overloaded optional argument and null guard

Open
#11,661 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug topic-overloads
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

Mypy fails to find type narrowings that use a combination of overloading and null checks.

To Reproduce

from __future__ import annotations
import typing as t


@t.overload
def func2(a: int,
          b: str,
          x: t.Callable[[int], int],
          y: t.Callable[[str], int]) -> tuple[int, int]: ...
@t.overload
def func2(a: int,
          b: str,
          x: t.Callable[[t.Union[int, str]], int]) -> tuple[int, int]: ...


def func2(a: int,
          b: str,
          x: t.Callable[[int], int],
          y: t.Optional[t.Callable[[str], int]] = None):
    y = y or x
    return x(a), y(b)


@t.overload
def func3(a: int,
          b: str,
          x: t.Callable[[int], int],
          y: t.Callable[[str], int]) -> tuple[int, int]: ...
@t.overload
def func3(a: int,
          b: str,
          x: t.Callable[[t.Union[int, str]], int]) -> tuple[int, int]: ...


def func3(a: int,
          b: str,
          x: t.Union[t.Callable[[int], int], t.Callable[[t.Union[int, str]], int]],
          y: t.Optional[t.Callable[[str], int]] = None):
    if y:
        y2 = y
    else:
        y2 = x
    return x(a), y2(b)


@t.overload
def func4(a: int,
          b: str,
          x: list[int],
          y: list[str]): ...
@t.overload
def func4(a: int,
          b: str,
          x: list[t.Union[int, str]]): ...


def func4(a: int,
          b: str,
          x: t.Union[list[int], list[t.Union[int, str]]],
          y: t.Optional[list[str]] = None):
    if y:
        y2 = y
    else:
        y2 = x
    x.append(a)
    y2.append(b)

Expected Behavior

My expectation is that in func2-func4, the Union type of x will always narrow down to support b as well as a when y is not supplied and x must be assigned to y/y2.

Actual Behavior

mypy==0.910:

$ mypy --show-error-codes _scratch.py 
_scratch.py:20: error: Incompatible types in assignment (expression has type "Union[Callable[[str], int], Callable[[int], int]]", variable has type "Optional[Callable[[str], int]]")  [assignment]
_scratch.py:21: error: "None" not callable  [misc]
_scratch.py:42: error: Incompatible types in assignment (expression has type "Union[Callable[[int], int], Callable[[Union[int, str]], int]]", variable has type "Callable[[str], int]")  [assignment]
_scratch.py:64: error: Incompatible types in assignment (expression has type "Union[List[int], List[Union[int, str]]]", variable has type "List[str]")  [assignment]

mypy==0.920.dev:

_scratch.py:20: error: Incompatible types in assignment (expression has type "Union[Callable[[str], int], Callable[[int], int]]", variable has type "Optional[Callable[[str], int]]")  [assignment]
_scratch.py:21: error: "None" not callable  [misc]
_scratch.py:42: error: Incompatible types in assignment (expression has type "Union[Callable[[int], int], Callable[[Union[int, str]], int]]", variable has type "Callable[[str], int]")  [assignment]
_scratch.py:64: error: Incompatible types in assignment (expression has type "Union[List[int], List[Union[int, str]]]", variable has type "List[str]")  [assignment]
_scratch.py:72: error: Cannot call function of unknown type  [operator]

Instead, no narrowing occurs and mypy declares the types incompatible, both when Callable is used and when a standard collection is used.

Your Environment

  • Mypy version used: 0.910 and 0.920+dev.cee5d3e9a29f43a10a8eeca760976657bf1689c9
  • Mypy command-line flags: --show-error-codes
  • Python version used: Python 3.8.0
  • Operating system and version: Microsoft Windows 10 [Version 10.0.19043.1348]

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the reported _scratch.py reproducer with the listed mypy versions and --show-error-codes, then compare the errors in func2, func3, and func4 with the expected narrowing behavior. Done means the overload and null-guard cases no longer produce the reported incompatible-assignment, None-call, or unknown-function errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.