python / python/mypy

type narrowing vs assert vs unreachability changed in PR #17818 (between mypy 1.11 and 1.12)

Open
#18,431 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

tldr, since PR #17817 release in mypy 1.12, and still this way in master 11cc21c4ae845ce19b0d108fa70b3b34a780f633, I have seen this behaviour change in the interaction of type narrowing, assert and unreachability checking:

# prereq  foo.x: int = 0
assert not foo.x           # narrows foo.x to Literal[0]
foo.reset()                # modifies `foo.x=4`, so `foo.x: Literal[0] = 4` (!)
assert foo.x               # at runtime this passes, but mypy reasons Literal[0] cannot pass this assert
# BAD: anything after here is warned as unreachable

mypy believes assert not foo.x and assert foo.x cannot both pass, mediated via type narrowing, across an opportunity for an object to change its own state.

To Reproduce

$ cat mp818.py 
from typing import Union
 
class Foo:
  def __init__(self):
    self.x: int = 0
 
  def reset(self):
    self.x=4
 
foo=Foo()
 
reveal_type(foo.x)
assert not foo.x
reveal_type(foo.x)
 
# SWAP THESE LINES TO MAKE MYPY HAPPY
# foo.x = 4
foo.reset()
 
reveal_type(foo.x)
assert foo.x
 
print(foo.x)
 
 
$ mypy mp818.py --warn-unreachable
 
# BAD, >= mypy PR #17818
 
 
mp818.py:5: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
mp818.py:12: note: Revealed type is "builtins.int"
mp818.py:14: note: Revealed type is "Literal[0]"
mp818.py:20: note: Revealed type is "Literal[0]"
mp818.py:23: error: Statement is unreachable  [unreachable]
Found 1 error in 1 file (checked 1 source file)

# GOOD, < PR #17818 *or* swap foo.reset() to foo.x=4
 
 
mp818.py:5: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
mp818.py:12: note: Revealed type is "builtins.int"
mp818.py:14: note: Revealed type is "builtins.int"
mp818.py:20: note: Revealed type is "builtins.int"
Success: no issues found in 1 source file

I'm encountering this in real life at https://github.com/Parsl/parsl/blob/34a2890c65f239bb145dca6af76bbdcc0443bc3e/parsl/tests/test_curvezmq.py#L310 where we assert that a socket is open, do something that should close it, then assert that it is really is closed.

Expected Behavior

I expected the behaviour to be the pre-17818 behaviour: if i assert on a field inside an object, don't carry reasoning from earlier asserts about that value across calls that might modify that value.

Actual Behavior

An assertion about the state of an object is carried across a mutation of that state and used to make incorrect reasoning about later asserts.

Your Environment

works with: mypy 1.11, 1.5.1
fails with: mypy 1.12, f6520c84746072e72e8a017963d3d2a3a9361771, master yesterday ccf05db67f6f99878c73eb902fc59a6f037b18a6

  • Mypy command-line flags:

mypy mp818.py --warn-unreachable (mp818.py is my reproducer above)

  • Mypy configuration options from mypy.ini (and other config files):

none

  • Python version used:

Python 3.12

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the mp818.py reproducer with mypy 1.11 and 1.12 using --warn-unreachable, then inspect PR #17818 and the related type-narrowing behavior. Compare how direct assignment and foo.reset() affect the revealed type after the first assertion. Done means the post-mutation assertion is analyzed without incorrectly reporting the following statement as unreachable.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.