python / python/typeshed

Shouldn't the return type of `__reversed__` be an `Iterable[T]` instead of `Iterator[T]`?

Open
#13,218 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
5.1k
Forks
2.1k
Avg merge
1d 19h
Merged PRs (30d)
82

Description

Currently the return type of Reversible.__reversed__ is typed as Iterator[T], which suggests that the returned instance must have both an __next__ and an __iter__ method.

https://github.com/python/typeshed/blob/0e9c9e1362959512a880abbf1275471b0d76924f/stdlib/typing.pyi#L446-L448

However, the following code seems to work fine at runtime:

from dataclasses import dataclass
from typing import Iterator, Iterable

@dataclass
class MyIter:
    _values: list[int]
    
    def __iter__(self) -> Iterator[int]:
        return iter(self._values)
    
    def __reversed__(self) -> Iterator[int]:
        return MyIter(self._values[::-1])

my_iter = MyIter([1, 2, 3])
for x in my_iter:
    print(x)
for x in reversed(my_iter):
    print(x)

Note that __reversed__ here returns a MyIter instance, which has an __iter__ method, but no __next__, i.e., it is an Iterable but not an Iterator. The Python interpreter seems to deal with that fine at runtime, i.e., it doesn't actually seem to need the __next__ method. This is slightly surprising, because the docs specify:

It should return a new iterator object that iterates over all the objects in the container in reverse order.

I.e., it doesn't use the word "iterable object".

Unfortunately, the current signature of __reversed__ means that this example does not type check: Obviously the type checker has to complain about the return MyIter(...) line, because MyIter is indeed only an Iterable (example on mypy playground):

main.py:13: error: Incompatible return value type (got "MyIter", expected "Iterator[int]")  [return-value]
main.py:13: note: "MyIter" is missing following "Iterator" protocol member:
main.py:13: note:     __next__
Found 1 error in 1 file (checked 1 source file)

Now I'm wondering if the signature should actually be def __reversed__(self) -> Iterable[int] to lessen the requirement and match the runtime behavior?

Note that simply changing the return type to Iterable on user side means that the return statement now type checks, but then all usages (reversed(my_iter)) stop to type check because the type checker will no longer consider MyIter as a valid Reversible (modified example on mypy playground):

main.py:18: error: No overload variant of "reversed" matches argument type "MyIter"  [call-overload]
main.py:18: note: Possible overload variants:
main.py:18: note:     def [_T] __new__(cls, Reversible[_T], /) -> reversed[_T]
main.py:18: note:     def [_T] __new__(cls, SupportsLenAndGetItem[_T], /) -> reversed[_T]
Found 1 error in 1 file (checked 1 source file)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Reversible definition in stdlib/typing.pyi at the linked lines, then compare the two mypy Playground examples in the issue. Determine a signature that accepts the shown reversed implementation while keeping reversed(my_iter) type-checking; done means both reported errors are resolved without weakening unrelated typing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.