python / python/mypy

Implicit TypeVar expansion of a Generic via another generic method

Offen
#6,933 7 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

bug priority-1-normal topic-type-variables
Vorherrschende Sprache
Python
Sterne
20.6k
Forks
3.3k
PR-Merge-Kennzahlen
PR-Kennzahlen ausstehend

Beschreibung

I really struggled to come up with a title for this, so I apologize for the ambiguity. I also considered "Spooky TypeVar action at a distance".

I'm investigating strategies for static type checking of apache beam, which is a streaming pipeline API.

Basically, I have a generic Transform with an In and Out type, and another generic Collection that wants to apply its type to the input type of Transform in order to produce a new Collection with the Transform's output type. These operations are repeated in a chain to form a pipeline:

Collection[None] -> Transform[None, A] -> Collection[A] -> Transform[A, B] -> Collection[B]

The catch is that a Transform's output type can be related to its input type via a TypeVar, and a Collection contains the concrete type of that input TypeVar. So when we call Collection.apply(transform) we want to resolve that chain of TypeVar dependencies to produce an expanded/concrete output type:

from typing import *

T = TypeVar('T')
InT = TypeVar('InT')
OutT = TypeVar('OutT')


class Transform(Generic[InT, OutT]):
    "Takes an input Collection and produces an output Collection"

    def __init__(self, fn: Callable[[InT], OutT]):
        self.fn = fn

    def call(self, arg: InT) -> OutT:
        return self.fn(arg)


def get_op(f: Callable[[InT], OutT]) -> Transform[InT, OutT]:
    "Get a Transform from a callable"
    return Transform(f)


class Collection(Generic[T]):
    "Collection of elements"

    def apply(self, op: Transform[T, OutT]) -> 'Collection[OutT]':
        """
        Apply the this Collection to a Transform and produce a new 
        output Collection
        """


# -- test:

def make_string(x: None) -> str:
    return 'foo'


def make_ones(x: T) -> Tuple[T, int]:
    return (x, 1)


c1: Collection = Collection()

str_op = get_op(make_string)
reveal_type(str_op)  # revealed: Transform[None, builtins.str*]

c2 = p1.apply(str_op)
reveal_type(c2)  # revealed: Collection[builtins.str*]

tuple_op = get_op(make_ones)
reveal_type(tuple_op)  # revealed: Transform[T`-1, Tuple[T`-1, builtins.int]]

c3 = c2.apply(tuple_op)
# Desired type is Collection[Tuple[builtins.str, builtins.int]]
reveal_type(c3)  # revealed: Collection[Tuple[T`-1, builtins.int]]

The crux of the issue is here:

class Collection(Generic[T]):
    "Collection of elements"

    def apply(self, op: Transform[T, OutT]) -> 'Collection[OutT]':
        """
        Apply the this Collection to a Transform and produce a new 
        output Collection
        """

My naive desire was that my Collection[builtins.str*] would apply its str type to T-1 of Transform[T-1, Tuple[T-1, builtins.int]] to produce Collection[Tuple[builtins.str, builtins.int]].

Instead I get:

error: Argument 1 to "apply" of "Collection" has incompatible type "Transform[T, Tuple[T, int]]"; expected "Transform[str, Tuple[T, int]]"

Is there any future universe where this is possible in mypy, or is it just too ambiguous?

Is it possible to write a mypy plugin to produce the desired result for Collection.apply?

  • edit: clarified relationship between Transform and Collection TypeVars

Beitragsleitfaden

Beitragsleitfaden öffnen

Erste Schritte

  1. Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
  3. Forke das Repository und arbeite in einem Branch.
  4. Öffne einen Pull Request, der die Issue-Nummer nennt.

Rechercherichtung

Beginne damit, das bereitgestellte generische Transform/Collection-Beispiel in mypy zu reproduzieren, wobei du dich auf Collection.apply und die Ergebnisse von reveal_type konzentrierst. Verfolge, wie der Checker die Beziehung zwischen den TypeVars verarbeitet, und bewerte, ob ein Plugin-Einstiegspunkt dies unterstützen kann; abgeschlossen bedeutet, einen klar abgegrenzten Implementierungspfad festzulegen oder zu dokumentieren, warum die angeforderte Inferenz nicht unterstützt wird.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python
Bereich
devtools
Issue-Typ
Feature
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Veraltet
Klarheit
Muss geklärt werden
Anfängerfreundlichkeit
35/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.