maximum arity of itertools.product() is hardcoded to 10. document, increase, or print a warning?
まだ誰も着手していません。
- 主要言語
- Python
- スター
- 5.1k
- フォーク
- 2.1k
- 平均マージ
- 1日 19時間
- マージ済み PR(30日)
- 82
説明
Hi,
for reasons that I can't control, I have to support a software that calls itertools.product() with an arity greater than 10. I use mypy (currently 1.15.0) and python 3.13.1, but the problem is largely independent from the software versions.
It took me a couple of days to realize that this case is not supported, because the maximum arity for itertools.product() is controlled by 10 manually written overloads, one for each arity.
For example, the overload for product()/10 is here:
The problem I faced is that the type checker (mypy 1.14.x and 1.15.x in my case), instead of clearly asserting that it could not work with arities greater than 10, spit out wrong deductions (see the example later on).
The obvious desiderata here would be not having to rely on any hardcoded overloads and support functions of arbitrary arity. I suppose this requires a more powerful type system, or checker, or both. In each case I do not expect it to happen in a short time, right?
However, it took me a couple of days to realize the existence of this limitation in the type deduction of itertools.product(), because I kept trying to find a bug in the code I have to maintain; I'd like to spare this suffering to future users.
Do you think at least one of the following is possible?
- find a way to cause a warning in the type checkers, so that users know that they are experiencing a limitation in the stubs/type checker and not in their code
- increment the number of supported overloads for
itertools.product()from 10 to 15, so that the limitation is hit more rarely. I have manually altereditertools.pyiin my venv, and it makes mypy happy (and cause misery to the idealists among us) - document the limitation on the maximum arity of
itertools.product(), for example in https://typing.readthedocs.io/
If 2. is acceptable, I am willing to submit a PR. With enough hand holding, I could also contribute point 3, if it helps.
Thank you very much.
Example of a program that breaks badly when iterable_11 is uncommented:
from itertools import product
from typing import Literal
def f(
iterable_01: tuple[Literal["one"]] | list[int],
iterable_02: list[Literal[2]],
iterable_03: list[Literal[3]],
iterable_04: list[Literal[4]],
iterable_05: list[Literal[5]],
iterable_06: list[Literal[6]],
iterable_07: list[Literal[7]],
iterable_08: list[Literal[8]],
iterable_09: list[Literal[9]],
iterable_10: list[Literal[10]] | tuple[None, Literal[10]],
# iterable_11: list[Literal[11]],
) -> None:
for (
elem_01,
elem_02,
elem_03,
elem_04,
elem_05,
elem_06,
elem_07,
elem_08,
elem_09,
elem_10,
# elem_11,
) in product(
iterable_01,
iterable_02,
iterable_03,
iterable_04,
iterable_05,
iterable_06,
iterable_07,
iterable_08,
iterable_09,
iterable_10,
# iterable_11,
):
reveal_type(elem_01)
reveal_type(elem_02)
reveal_type(elem_03)
reveal_type(elem_04)
reveal_type(elem_05)
reveal_type(elem_06)
reveal_type(elem_07)
reveal_type(elem_08)
reveal_type(elem_09)
reveal_type(elem_10)
Type checking with 10 arguments and correct type deduction:
$ mypy itertools_product.py
itertools_product.py:43: note: Revealed type is "Literal['one']"
itertools_product.py:44: note: Revealed type is "Literal[2]"
itertools_product.py:45: note: Revealed type is "Literal[3]"
itertools_product.py:46: note: Revealed type is "Literal[4]"
itertools_product.py:47: note: Revealed type is "Literal[5]"
itertools_product.py:48: note: Revealed type is "Literal[6]"
itertools_product.py:49: note: Revealed type is "Literal[7]"
itertools_product.py:50: note: Revealed type is "Literal[8]"
itertools_product.py:51: note: Revealed type is "Literal[9]"
itertools_product.py:52: note: Revealed type is "Union[Literal[10], None]"
Success: no issues found in 1 source file
Type checkng with 11 arguments and incorrect type deduction (this is random: sometimes mypy writes builtins.object, instead):
$ mypy itertools_product.py
itertools_product.py:43: note: Revealed type is "Union[Literal['one'], builtins.int, None]" <-- wider, but wrong
itertools_product.py:44: note: Revealed type is "Union[Literal['one'], builtins.int, None]" <-- completely wrong
itertools_product.py:45: note: Revealed type is "Union[Literal['one'], builtins.int, None]" [...]
itertools_product.py:46: note: Revealed type is "Union[Literal['one'], builtins.int, None]" [...]
itertools_product.py:47: note: Revealed type is "Union[Literal['one'], builtins.int, None]" [...]
itertools_product.py:48: note: Revealed type is "Union[Literal['one'], builtins.int, None]" [...]
itertools_product.py:49: note: Revealed type is "Union[Literal['one'], builtins.int, None]" [...]
itertools_product.py:50: note: Revealed type is "Union[Literal['one'], builtins.int, None]" [...]
itertools_product.py:51: note: Revealed type is "Union[Literal['one'], builtins.int, None]" [...]
itertools_product.py:52: note: Revealed type is "Union[Literal['one'], builtins.int, None]" <-- completely wrong
Success: no issues found in 1 source file
コントリビューションガイド
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
調査の方向性
Issue でリンクされている product() のオーバーロード周辺の stdlib/itertools.pyi から始め、続いて提供されている 10 引数および 11 引数の例を mypy で実行して、推論の違いを再現します。選択肢としてオーバーロードを拡張するか、制限を文書化するか、警告を追求するかを決める前に、typing.readthedocs.io に記載されている場所と議論を確認します。完了とは、選択した対応策について合意され、その動作が実証されていることを意味します。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- python
- 領域
- tooling
- issue の種類
- 機能追加
- 難易度
- 3/5
- 見積もり時間
- 1〜2日
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 42/100