type narrowing vs assert vs unreachability changed in PR #17818 (between mypy 1.11 and 1.12)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 20.6k
- Forks
- 3.3k
- PR merge metrics
- PR metrics pending
Description
Bug Report
tldr, since PR #17817 release in mypy 1.12, and still this way in master 11cc21c4ae845ce19b0d108fa70b3b34a780f633, I have seen this behaviour change in the interaction of type narrowing, assert and unreachability checking:
# prereq foo.x: int = 0
assert not foo.x # narrows foo.x to Literal[0]
foo.reset() # modifies `foo.x=4`, so `foo.x: Literal[0] = 4` (!)
assert foo.x # at runtime this passes, but mypy reasons Literal[0] cannot pass this assert
# BAD: anything after here is warned as unreachable
mypy believes assert not foo.x and assert foo.x cannot both pass, mediated via type narrowing, across an opportunity for an object to change its own state.
To Reproduce
$ cat mp818.py
from typing import Union
class Foo:
def __init__(self):
self.x: int = 0
def reset(self):
self.x=4
foo=Foo()
reveal_type(foo.x)
assert not foo.x
reveal_type(foo.x)
# SWAP THESE LINES TO MAKE MYPY HAPPY
# foo.x = 4
foo.reset()
reveal_type(foo.x)
assert foo.x
print(foo.x)
$ mypy mp818.py --warn-unreachable
# BAD, >= mypy PR #17818
mp818.py:5: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked]
mp818.py:12: note: Revealed type is "builtins.int"
mp818.py:14: note: Revealed type is "Literal[0]"
mp818.py:20: note: Revealed type is "Literal[0]"
mp818.py:23: error: Statement is unreachable [unreachable]
Found 1 error in 1 file (checked 1 source file)
# GOOD, < PR #17818 *or* swap foo.reset() to foo.x=4
mp818.py:5: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked]
mp818.py:12: note: Revealed type is "builtins.int"
mp818.py:14: note: Revealed type is "builtins.int"
mp818.py:20: note: Revealed type is "builtins.int"
Success: no issues found in 1 source file
I'm encountering this in real life at https://github.com/Parsl/parsl/blob/34a2890c65f239bb145dca6af76bbdcc0443bc3e/parsl/tests/test_curvezmq.py#L310 where we assert that a socket is open, do something that should close it, then assert that it is really is closed.
Expected Behavior
I expected the behaviour to be the pre-17818 behaviour: if i assert on a field inside an object, don't carry reasoning from earlier asserts about that value across calls that might modify that value.
Actual Behavior
An assertion about the state of an object is carried across a mutation of that state and used to make incorrect reasoning about later asserts.
Your Environment
works with: mypy 1.11, 1.5.1
fails with: mypy 1.12, f6520c84746072e72e8a017963d3d2a3a9361771, master yesterday ccf05db67f6f99878c73eb902fc59a6f037b18a6
- Mypy command-line flags:
mypy mp818.py --warn-unreachable (mp818.py is my reproducer above)
- Mypy configuration options from
mypy.ini(and other config files):
none
- Python version used:
Python 3.12
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the mp818.py reproducer with mypy 1.11 and 1.12 using --warn-unreachable, then inspect PR #17818 and the related type-narrowing behavior. Compare how direct assignment and foo.reset() affect the revealed type after the first assertion. Done means the post-mutation assertion is analyzed without incorrectly reporting the following statement as unreachable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100