python / python/mypy

TypeGuard typechecking on `filter` is too strict

Open
#12,682 10 comments 34 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug topic-overloads topic-type-context topic-typeguard-typeis
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

NB: This may be more properly an issue in Typeshed, or require changes in that project to fix, but our team encountered it via MyPy.

The type annotations on builtins.filter cause correct and idiomatic usages to fail to typecheck:

To Reproduce

# filter-mypy.py
path_parts: List[Optional[str]] = ["foo", None, "bar"]

"/".join(filter(None, path_parts))
"/".join(filter(bool, path_parts))
"/".join(filter(lambda s: isinstance(s, str), path_parts))

Expected:

$> mypy filter-mypy.py
Success: no issues found in 1 source file

Actual result:

$> mypy filter-mypy.py
filter-mypy.py:6: error: Argument 1 to "filter" has incompatible type "Type[bool]"; expected "Callable[[Optional[str]], TypeGuard[str]]"
filter-mypy.py:7: error: Argument 1 to "filter" has incompatible type "Callable[[Any], bool]"; expected "Callable[[Optional[str]], TypeGuard[str]]"
Found 2 errors in 1 file (checked 1 source file)

Your Environment

  • Mypy version used:
$> mypy --version
mypy 0.950 (compiled: yes)
  • Python version used:
$> python --version
Python 3.10.4
  • Operating system and version: Ubuntu 20.04

The following is what my editor (VSCode) pulls up when I investigate the type hints for filter, though I doubt this is the same file that MyPy itself is consulting:

class filter(Iterator[_T], Generic[_T]):
    @overload
    def __init__(self, __function: None, __iterable: Iterable[_T | None]) -> None: ...
    @overload
    def __init__(self, __function: Callable[[_S], TypeGuard[_T]], __iterable: Iterable[_S]) -> None: ...
    @overload
    def __init__(self, __function: Callable[[_T], Any], __iterable: Iterable[_T]) -> None: ...

Note that both offending lines conform to the third type signature, but fail the second.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the report with the provided filter-mypy.py example and inspect the builtins.filter overloads shown in the issue, then determine whether the fix belongs in mypy or typeshed. Done means the None, bool, and isinstance usages typecheck as expected without weakening unrelated filter checking.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.