python / python/mypy

Problem with @overload and Iterator/Generator

Open
#17,927 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

See last line (Playground):

from collections.abc import Iterator, Generator
from typing import Any, Iterable, TypeVar, overload
from typing_extensions import assert_type

T = TypeVar("T")
IterableT = TypeVar("IterableT", bound=Iterable)


@overload
def read_iterator(iterable: Iterator[T]) -> list[T]: ...


@overload
def read_iterator(iterable: IterableT) -> IterableT: ...


def read_iterator(iterable: Iterable) -> Iterable:
    if isinstance(iterable, Iterator):
        return list(iterable)
    else:
        return iterable


def _create_generator() -> Generator[int, Any, None]:
    for i in range(3):
        yield i


def test_read_iterator() -> None:
    an_iterator = iter(range(3))
    assert_type(read_iterator(an_iterator), list[int])

    a_range, a_list, a_tuple, a_set, a_dict = range(3), [1, 2, 3], (1, 2, 3), {1, 2, 3}, {"a": 1}
    assert_type(read_iterator(a_range), range)
    assert_type(read_iterator(a_list), list[int])
    assert_type(read_iterator(a_tuple), tuple[int, int, int])
    assert_type(read_iterator(a_set), set[int])
    assert_type(read_iterator(a_dict), dict[str, int])
    
    a_generator = _create_generator()
    assert_type(a_generator, Generator[int, Any, None])
    assert_type(read_iterator(a_generator), list[int])  # error: Expression is of type "Any", not "list[int]"  [assert-type]

Mypy doesn't like this code:

error: Expression is of type "Any", not "list[int]"  [assert-type]

Please remember that Generator is a subtype of Iterator:

>>> isinstance(a_generator, Iterator)
True

I think this is a bug in Mypy. The Generator should match the signature of def read_iterator(iterable: Iterator[T]) -> list[T]


Versions:

> python -V             
Python 3.11.1

> mypy --version
mypy 1.11.2 (compiled: yes)

❯ system_profiler SPSoftwareDataType
Software:

    System Software Overview:

      System Version: macOS 13.6.7 (22G720)
      Kernel Version: Darwin 22.6.0

Related: https://github.com/python/typing/issues/253

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the overload-resolution case from the linked mypy-play Playground, focusing on how Generator and Iterator are matched and how the TypeVar return type is inferred. Done means the final assert_type(read_iterator(a_generator), list[int]) passes without the reported error, while the other assertions remain valid.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.