python / python/mypy

Callable with Iterator[T] return type and default function with compatible return type, errors

Open
#10,504 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

I am trying to add some type hints to a module that takes a filename and reads it with a caller supplied reader (most common csv.reader and csv.DictReader). I noticed when I do not supply a default, mypy is fine, however, when I do, it complains about incompatibility.

To Reproduce

import csv
from typing import Callable, Iterable, Iterator, Text, TypeVar

T = TypeVar('T')

def foo(x: str, reader: Callable[[Iterable[Text]], Iterator[T]]) -> Iterator[T]:
    return reader(x)


# Is fine, resolves to Iterator[List[str]]
x = foo('bar.txt', csv.reader)


# Complains that _reader is not compatible with Iterator[T]
def baz(x: str, reader: Callable[[Iterable[Text]], Iterator[T]] = csv.reader) -> Iterator[T]:
    return reader(x)

When I look at the _reader definition I see

class _reader(Iterator[List[str]]):
    dialect: Dialect
    line_num: int
    if sys.version_info >= (3, 0):
        def __next__(self) -> List[str]: ...
    else:
        def next(self) -> List[str]: ...

So it is a subclass of Iterator[List[str]] which should be compatible with Iterator[T]

Expected Behavior

No mypy error is shown

Actual Behavior

mypy complains about using csv.reader as the default value of the reader argument due to it not matching _reader.

Your Environment

  • Mypy version used: 0.812
  • Mypy command-line flags:
  • Mypy configuration options from mypy.ini (and other config files):
[mypy]
# Most copied from https://github.com/python/mypy/blob/675ae1afc8a5b0a9a515292c7e9d6bc2a38887be/mypy_self_check.ini
disallow_untyped_calls = True
disallow_untyped_defs = True
disallow_incomplete_defs = True
check_untyped_defs = True
disallow_subclassing_any = True
warn_no_return = True
strict_optional = True
strict_equality = True
no_implicit_optional = True
disallow_any_generics = True
disallow_any_unimported = True
warn_redundant_casts = True
warn_unused_ignores = True
warn_unused_configs = True
show_traceback = True
show_error_codes = True

# Should be removed if we use typehints everywhere
follow_imports = silent
python_version = 3.9

[mypy-memory_profiler.*]
ignore_missing_imports = True
  • Python version used: 3.9.1
  • Operating system and version: Ubuntu 20.10

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the supplied Python snippet with mypy 0.812 and compare the generic reader parameter in foo with the default in baz. Trace how mypy checks a default callable against a generic return type. Done means baz accepts csv.reader without an incompatibility error while preserving the existing behavior for foo.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.