python / python/mypy

Optionally disallow membership checking on Iterator types.

オープン
#10,465 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

feature
主要言語
Python
スター
20.6k
フォーク
3.3k
PR マージ指標
PR 指標を取得中

説明

Feature

In a production system, we recently encountered a bug where a containment check was executed on a generator

The gist was something like the following.

items = (foo.bar for foo in all_the_foos)
for x in y:
    if x in items:
        do_sparkle_hands_stuff()

From a typing perspective, I think this is perfectly fine -- You certainly can do a containment check on a generator -- or any Iterable (or seemingly any object with a __getitem__ though I'm not sure there is an ABC for that) according to the language reference

For objects that don’t define contains(), the membership test first tries iteration via iter(), then the old sequence iteration protocol via getitem(), see this section in the language reference.

The defined behavior here can lead to subtle bugs for Iterator types since the iterator can get consumed by the first element that isn't in the iterator and then all subsequent items are guaranteed to not be in. It seems like it would be nice if mypy could alert us to the presence of these potential bugs.

It's worth asking if there are valid use-cases where we would want this behavior?

the potential "one-shot containment" question could be implemented as:

y in (x.foo for x in foos)

But that's probably more clearly written as:

any(y == x.foo for x in foos)

I suppose that there could be potential use-cases where the side-effect of consuming the iterable might be desired ('m thinking about checking things in a pair of sorted iterators to do some kind of linear containment check with break at first miss)

for y in sorted_y:
    if y in sorted_x:  # sorted_x is an iterator
        ...

But this really seems like a stretch-case that probably doesn't happen and if it did a targeted type: ignore statement would probably be fine.

It's also probably worth discussing whether this should become a stronger statement by disallowing membership checking on Iterable types -- then we'd have to have a way to cut off to make sure that subclasses of Iterable (e.g. Collection and onward) would still support membership checking. That one might also lead to more false-positives.

Pitch

I suspect that people might want to hide this behavior behind a command-line flag since it is technically type-safe already, but I'd also be fine if this was just always on, or enabled as part of --strict, etc.

コントリビューションガイド

コントリビューションガイドを開く

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

調査の方向性

まず、ここで提起されているコマンドラインおよび strict-mode の代替案を含め、Iterator、Iterable、Collection に対して提案されている動作を比較します。最初に、受け入れるポリシーとその誤検知とのトレードオフを定義します。完了とは、選択した動作が仕様化・実装され、generator のメンバーシップおよびサポート対象の例外について回帰ケースでカバーされていることを意味します。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
python
領域
tooling
issue の種類
機能追加
難易度
5/5
見積もり時間
1週間以上
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
38/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。