python / python/mypy

Indexing TypeVars / need a workaround for higher-kindedness

Open
#6,066 7 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

feature topic-type-variables
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

I'm using mypy 0.630.

I have a container, which can contain various types, some of which contain further types. I want the root of the container to be able to return grandchild types, or failing that to be able to return fully-specified child types. My first thought doesn't work:

from typing import TypeVar, Generic, List

ContainedType = TypeVar('ContainedType', bound='ContainedCls')
SubContained = TypeVar('SubContained')
T = TypeVar('T')

class Root_wrong(Generic[ContainedType[SubContained]]):
    def __init__(self, ct):
        # type: (ContainedType[SubContained]) -> None
        self.contained = ct

    def iter_subcontained(self):
        # type: () -> List[SubContained]
        return self.contained.items

class ContainedCls(Generic[T]):
    def __init__(self, items):
        # type: (List[T]) -> None
        self.items = items

Because TypeVars can't be indexed.

This also doesn't work, because iter_subcontained can't be typed:

class Root(Generic[ContainedType]):
    def __init__(self, ct):
        # type: (ContainedType) -> None
        # I know because of the bound that ContainedType will have a parameter of its own, but I
        # can't refer to it in the `class Root` declaration because TypeVars can't be indexed.
        # and the index to Generic has to be a typevar
        self.contained = ct

    def iter_subcontained(self):
        # what goes here?
        return self.contained.items


reveal_type(Root(ContainedCls([1])).iter_subcontained())  # Any

Alas, this also doesn't work:

class Root2(Generic[ContainedType, SubContained]):
    def __init__(self, ct):
        # type: (ContainedType) -> None
        # I know because of the bound that ContainedType will have a parameter of its own, but I
        # can't refer to it in the `class Root` declaration because TypeVars can't be indexed.
        # and the index to Generic has to be a typevar
        self.contained = ct

    def iter_subcontained(self):
        # type: () -> List[SubContained]
        return self.contained.items


reveal_type(Root2(ContainedCls([1])).iter_subcontained())  # builtins.list[<nothing>]

# not matching is not an error
reveal_type(Root2[ContainedType[int], str](ContainedCls([1])).iter_subcontained())  # builtins.list[str]

This almost works:

class Root3(Generic[T]):
    def __init__(self, ct1, ct2):
        # type: (ContainedCls[T], ContainedCls[T]) -> None
        self.contained = ct1
        self.contained2 = ct2

    def iter_subcontained(self):
        # type: () -> List[T]
        return self.contained.items


class ContainedClsPrime(ContainedCls):
    pass


# no way to ensure that ct1 and ct2 are the same container type
# in fact, they aren't even constrained to have the same T!
x = Root3[int](ContainedCls([1]), ContainedClsPrime(['definitely not an int']))
reveal_type(x.iter_subcontained())  # builtins.list[str]
reveal_type(x.contained2.items[0])  # builtins.int* !!!

But it doesn't constrain the two ContainedCls[T]s to be the same ContainedCls subtype, and it doesn't (this really surprised me) even constrain them to be containing the same T! It also seems wrong to be able to make Root3 generic only in the leaf types, and not in any of the intermediate types.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the four examples in the issue with mypy and compare the revealed types against the stated expectations. Read the generic and TypeVar handling relevant to indexed TypeVars and higher-kinded types. Done means establishing a supported workaround or a clearly scoped implementation path for preserving nested container types.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.