python / python/mypy

=| and &= allowed for immutable set ABC

Open
#14,741 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

mypy seems to allow |= and &= for collections.abc.Set, which is an immutable set type. Its documentation declares that it doesn't offer the __ior__ and __iand__ methods. Indeed, mypy rejects the use of these methods directly, but it does allow |= and &=.

A consequence of this is that because collections.abc.Set is covariant in its element type, breaking out of its immutability allows breaking the Liskov substitution principle.

I've checked both the collections.abc and the typeshed implementation of the relevant types and as far as I can make out they seem to be defined in line with the documentation, so I assume this is a mypy issue, but I could be wrong about that.

To Reproduce

https://mypy-play.net/?mypy=latest&python=3.11&gist=ca538f567c30d410778f87454f9d3494

import collections.abc
from typing import TYPE_CHECKING

myset: collections.abc.Set[int] = {1}
if TYPE_CHECKING:
    reveal_type(myset)
alias: collections.abc.Set[int] = myset
myset |= {2}
# proof that the set object has been modified despite being declared immutable
assert alias == {1, 2}

# Consequence: break Liskov

class Parent: pass
class Child(Parent): pass

def f(s: collections.abc.Set[Parent]) -> None:
    # s should be immutable and yet we can extend it
    s |= {Parent()}

children: collections.abc.Set[Child] = {Child()}
# mypy allows this because the immutable `Set` is covariant in its element type.
# This is safe because, it being immutable, the method can not add elements to it.
# Except that with this bug it can, adding an `Parent` to a `Set[Child]`
f(children)
if TYPE_CHECKING:
    reveal_type(children)
# This should never happen
assert any(isinstance(child, Parent) for child in children)

Expected Behavior

I expected mypy to reject |= and &= for collections.abc.Set and only allow it for collections.abc.MutableSet.

Actual Behavior

Mypy finds no issues with the snippet:

main.py:6: note: Revealed type is "typing.AbstractSet[builtins.int]"
main.py:27: note: Revealed type is "typing.AbstractSet[__main__.Child]"
Success: no issues found in 1 source file

Your Environment

  • Mypy version used: 0.931 locally, 1.0.0 on mypy-play.net
  • Mypy command-line flags: none
  • Mypy configuration options from mypy.ini (and other config files): none
  • Python version used: 3.9 locally, 3.11 on mypy-play.net

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked mypy-play reproduction and trace how augmented assignments are checked for collections.abc.Set versus MutableSet. Done means |= and &= are rejected for immutable Set while remaining allowed for MutableSet, with regression coverage for the reported examples.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.