python / python/mypy

Optionally disallow membership checking on Iterator types.

Offen
#10,465 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

feature
Vorherrschende Sprache
Python
Sterne
20.6k
Forks
3.3k
PR-Merge-Kennzahlen
PR-Kennzahlen ausstehend

Beschreibung

Feature

In a production system, we recently encountered a bug where a containment check was executed on a generator

The gist was something like the following.

items = (foo.bar for foo in all_the_foos)
for x in y:
    if x in items:
        do_sparkle_hands_stuff()

From a typing perspective, I think this is perfectly fine -- You certainly can do a containment check on a generator -- or any Iterable (or seemingly any object with a __getitem__ though I'm not sure there is an ABC for that) according to the language reference

For objects that don’t define contains(), the membership test first tries iteration via iter(), then the old sequence iteration protocol via getitem(), see this section in the language reference.

The defined behavior here can lead to subtle bugs for Iterator types since the iterator can get consumed by the first element that isn't in the iterator and then all subsequent items are guaranteed to not be in. It seems like it would be nice if mypy could alert us to the presence of these potential bugs.

It's worth asking if there are valid use-cases where we would want this behavior?

the potential "one-shot containment" question could be implemented as:

y in (x.foo for x in foos)

But that's probably more clearly written as:

any(y == x.foo for x in foos)

I suppose that there could be potential use-cases where the side-effect of consuming the iterable might be desired ('m thinking about checking things in a pair of sorted iterators to do some kind of linear containment check with break at first miss)

for y in sorted_y:
    if y in sorted_x:  # sorted_x is an iterator
        ...

But this really seems like a stretch-case that probably doesn't happen and if it did a targeted type: ignore statement would probably be fine.

It's also probably worth discussing whether this should become a stronger statement by disallowing membership checking on Iterable types -- then we'd have to have a way to cut off to make sure that subclasses of Iterable (e.g. Collection and onward) would still support membership checking. That one might also lead to more false-positives.

Pitch

I suspect that people might want to hide this behavior behind a command-line flag since it is technically type-safe already, but I'd also be fine if this was just always on, or enabled as part of --strict, etc.

Beitragsleitfaden

Beitragsleitfaden öffnen

Erste Schritte

  1. Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
  3. Forke das Repository und arbeite in einem Branch.
  4. Öffne einen Pull Request, der die Issue-Nummer nennt.

Rechercherichtung

Beginne mit dem Vergleich des vorgeschlagenen Verhaltens für Iterator, Iterable und Collection, einschließlich der hier aufgeworfenen Alternativen für die Kommandozeile und den strict-mode. Definiere zuerst die akzeptierte Richtlinie und die Abwägungen bei false positives; abgeschlossen bedeutet, dass das gewählte Verhalten spezifiziert, implementiert und durch Regressionstests für die Zugehörigkeit von generator und alle unterstützten Ausnahmen abgedeckt ist.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python
Bereich
tooling
Issue-Typ
Feature
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
38/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.