python / python/mypy

Allow `types.GeneratorType` to be used instead of `collections.abc.Generator` as the return type of generator functions

Open
#20,522 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature topic-runtime-semantics
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Feature

Currently, mypy only allows collections.abc.Generator and its supertypes as the return type of generator functions, and as a result, it cannot infer the correct return type for awaitable objects that define the __await__() method via types.GeneratorType:

import asyncio

from collections.abc import Generator
from types import GeneratorType
from typing import Never, reveal_type


class A:
    def __await__(self) -> GeneratorType[Never, Never, bool]:
        return True
        yield  # generator definition


class B:
    def __await__(self) -> Generator[Never, Never, bool]:
        return True
        yield  # generator definition


async def main() -> None:
    reveal_type(await A())  # revealed type is "Any"
    reveal_type(await B())  # revealed type is "builtins.bool"

    reveal_type(A().__await__().gi_running)  # revealed type is "builtins.bool"
    reveal_type(B().__await__().gi_running)  # has no attribute "gi_running"


if __name__ == "__main__":
    asyncio.run(main())
example.py:9: error: The return type of a generator function should be "Generator" or one of its supertypes  [misc]
example.py:21: note: Revealed type is "Any"
example.py:22: note: Revealed type is "builtins.bool"
example.py:24: note: Revealed type is "builtins.bool"
example.py:25: error: "Generator[Never, Never, bool]" has no attribute "gi_running"  [attr-defined]
example.py:25: note: Revealed type is "Any"
Found 2 errors in 1 file (checked 1 source file)

I propose allowing the use of types.GeneratorType with subsequent inference of the correct return type.

Pitch

First, pyright already has the expected behavior:

/home/user/workspace/example.py
  /home/user/workspace/example.py:21:17 - information: Type of "await A()" is "bool"
  /home/user/workspace/example.py:22:17 - information: Type of "await B()" is "bool"
  /home/user/workspace/example.py:24:17 - information: Type of "A().__await__().gi_running" is "bool"
  /home/user/workspace/example.py:25:17 - information: Type of "B().__await__().gi_running" is "Unknown"
  /home/user/workspace/example.py:25:33 - error: Cannot access attribute "gi_running" for class "Generator[Never, Never, bool]"
    Attribute "gi_running" is unknown (reportAttributeAccessIssue)
1 error, 0 warnings, 4 informations

Second, this relates to the eternally controversial python/typing#1480. In python/typeshed#10816, false properties related to native objects were removed in favor of the types module. You could say that we can use isinstance/inspect.isgenerator to narrow collections.abc.Generator down to types.GeneratorType, but... it is not that simple:

    x = B().__await__()
    reveal_type(x)  # typing.Generator[Never, Never, builtins.bool]

    if isgenerator(x):
        reveal_type(x)  # types.GeneratorType[Any, Any, Any]

    if isinstance(x, GeneratorType):
        reveal_type(x)  # types.GeneratorType[Any, Any, Any]

I think that allowing the use of types.GeneratorType would make it easier to inspect native generator objects. Especially since collections.abc.Generator can also be used to define custom generator types.

Related: #18635.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the example.py case in the issue and compare mypy's current diagnostics with the expected pyright behavior. The work is done when types.GeneratorType is accepted as a generator-function return type and the shown await and generator attribute types are inferred as expected.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.