python / python/mypy

Overlapping overloads in second-order functions cause returned value to be Any

Aperta
#13,436 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

bug topic-overloads
Lingua principale
Python
Stelle
20.6k
Fork
3.3k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

Bug Report

Changing a parameter of a function from a determined type (e.g. int, str, etc.) to Any makes the return type of a second-order function applied on the initial function Any in some cases.

I originally found this issue using the pytest library (see in the wild section below).

To Reproduce
from typing import Any, Callable, overload, TypeVar

C = TypeVar("C", bound=Callable[..., object])

#
## the second-order function
#

@overload
def wrapper(arg: C) -> C:
    ...

@overload
def wrapper(arg: object) -> object:
    ...

def wrapper(arg: object) -> object:
    return arg  # we don't care

#
## the "working" case
#

def case_a(x: int) -> None:
    pass

reveal_type(case_a)           # Revealed type is "def (x: builtins.int)"
reveal_type(wrapper(case_a))  # Revealed type is "def (x: builtins.int)"

#
## the "issue" case: only the type of the `x` parameter has changed
#

def case_b(x: Any) -> None:
    pass

reveal_type(case_b)           # Revealed type is "def (x: Any)
reveal_type(wrapper(case_b))  # Revealed type is "Any"
Expected Behavior

reveal_type should give the same result for case_b and wrapper(case_b) (just like it did for case_a and wrapper(case_a)).

Actual Behavior

wrapper(case_b) is thought to have type Any

In the wild (pytest)
from typing import Any, Callable, TYPE_CHECKING, Union

import pytest

@pytest.mark.parametrize("obj", (42, "42"))
def test_bug0(obj: Union[int, str]) -> None:
    assert len(str(obj)) < 20

@pytest.mark.parametrize("obj", (42, "42"))
def test_bug1(obj: Any) -> None:
    assert len(str(obj)) < 20

@pytest.mark.parametrize("obj", (int, float))
def test_bug2(obj: Callable[[str], float]) -> None:
    assert len(obj.__name__) < 20

@pytest.mark.parametrize("obj", (int, float))
def test_bug3(obj: Callable[..., Any]) -> None:
    assert len(obj.__name__) < 20

if TYPE_CHECKING:
    reveal_type(test_bug0)  # Revealed type is "def (obj: Union[builtins.int, builtins.str])"
    reveal_type(test_bug1)  # Revealed type is "Any"
    reveal_type(test_bug2)  # Revealed type is "def (fct: def (builtins.str) -> builtins.float)"
    reveal_type(test_bug3)  # Revealed type is "Any"

We can see that test_bug0 and test_bug2 are well-typed, but test_bug1 and test_bug3 are typed as Any, which cause an error each (with --disallow-any-decorated):

$ mypy --disallow-any-decorated bug.py
bug.py:10: error: Function is untyped after decorator transformation
bug.py:18: error: Function is untyped after decorator transformation
bug.py:22: note: Revealed type is "def (obj: Union[builtins.int, builtins.str])"
bug.py:23: note: Revealed type is "Any"
bug.py:24: note: Revealed type is "def (fct: def (builtins.str) -> builtins.float)"
bug.py:25: note: Revealed type is "Any"
Found 2 errors in 1 file (checked 1 source file)

Here is the relevant parts of the source code of pytest for easy reference:

Markable = TypeVar("Markable", bound=Union[Callable[..., object], type])

class MarkDecorator:
    # Type ignored because the overloads overlap with an incompatible
    # return type. Not much we can do about that. Thankfully mypy picks
    # the first match so it works out even if we break the rules.
    @overload
    def __call__(self, arg: Markable) -> Markable:  # type: ignore[misc]
        pass

    @overload
    def __call__(self, *args: object, **kwargs: object) -> "MarkDecorator":
        pass

    def __call__(self, *args: object, **kwargs: object):
        pass  # redacted

# pytest.mark.parametrize returns a MarkDecorator instance
Your Environment
  • Mypy version used: 0.971
  • Mypy command-line flags: none, except for the pytest case (see above)
  • Mypy configuration options from mypy.ini (and other config files): none
  • Python version used: 3.10
  • Operating system and version: Arch Linux

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Direzione di ricerca

Inizia eseguendo il riproduttore fornito di wrapper/case_a/case_b e l'esempio pytest MarkDecorator.call da src/_pytest/mark/structures.py. Traccia la risoluzione degli overload per il callable con parametro Any e confronta i risultati di reveal_type. Il lavoro è completato quando wrapper(case_b), test_bug1 e test_bug3 mantengono le proprie firme callable invece di diventare Any, mentre i casi esistenti rimangono invariati.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
compilers
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.