`mypy` behavior differs when run on a file directly vs a test
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 20.6k
- Forks
- 3.3k
- PR merge metrics
- PR metrics pending
Description
Bug Report
Likely due to some test stubs inconsistency, mypy behaviour in test deviates from results of running full mypy on a file.
This is a follow-up for #18675.
To Reproduce
Add the following test to check-enum.test:
[case testDemo]
# flags: --python-version 3.13 --warn-unreachable
from enum import Enum, member, nonmember
from typing import Literal, Never
def assert_never(_: Never) -> Never: ...
class E2(Enum):
@member
def C() -> None: ... # E: Method must have at least one argument. Did you forget the "self" argument? [misc]
c: Literal[E2.C] # E: Parameter 1 of Literal[...] is invalid [valid-type]
def check_2(e: E2) -> None:
match e:
case E2.C:
pass
case other:
assert_never(other) # E: Argument 1 to "assert_never" has incompatible type "E2"; expected "Never" [arg-type]
if e is E2.C:
pass
else:
assert_never(e) # E: Argument 1 to "assert_never" has incompatible type "<subclass of "enum.member[Callable[[], None]]" and "__main__.E2">"; expected "Never" [arg-type]
[builtins fixtures/enum.pyi]
Now copy the test code to a separate file and run mypy on it.
Expected Behavior
Test and mypy outputs are identical.
Actual Behavior
$ pytest mypy/test/testcheck.py::TypeCheckSuite::check-enum.test -k testDemo
=========================================================================================================== test session starts ============================================================================================================
platform linux -- Python 3.12.6, pytest-8.3.3, pluggy-1.5.0
rootdir: /home/stas/Documents/Work/mypy
configfile: pyproject.toml
plugins: cov-5.0.0, xdist-3.6.1
2 workers [1 item]
F [100%]
================================================================================================================= FAILURES =================================================================================================================
_________________________________________________________________________________________________________________ testDemo _________________________________________________________________________________________________________________
[gw0] linux -- Python 3.12.6 /home/stas/Documents/Work/mypy/.venv12/bin/python3
data: /home/stas/Documents/Work/mypy/test-data/unit/check-enum.test:2516:
Failed: Unexpected type checker output (/home/stas/Documents/Work/mypy/test-data/unit/check-enum.test, line 2516)
----------------------------------------------------------------------------------------------------------- Captured stderr call -----------------------------------------------------------------------------------------------------------
Expected:
main:9: error: Method must have at least one argument. Did you forget the "self" argument? [misc] (diff)
main:11: error: Parameter 1 of Literal[...] is invalid [valid-type] (diff)
main:18: error: Argument 1 to "assert_never" has incompatible type "E2"; expected "Never" [arg-type] (diff)
main:23: error: Argument 1 to "assert_never" has incompatible type "<subclass of "enum.member[Callable[[], None]]" and "__main__.E2">"; expected "Never" [arg-type] (diff)
Actual:
main:9: error: Method must have at least one argument. Did you forget the "self" argument? (diff)
main:11: error: Parameter 1 of Literal[...] is invalid (diff)
main:18: error: Argument 1 to "assert_never" has incompatible type "Union[member[Callable[[], None]], E2]"; expected "Never" (diff)
Alignment of first line difference:
E: ...the "self" argument? [misc]
A: ...the "self" argument?
^
Update the test output using --update-data (implies -n0; you can additionally use the -k selector to update only specific tests)
========================================================================================================= short test summary info ==========================================================================================================
FAILED mypy/test/testcheck.py::TypeCheckSuite::check-enum.test::testDemo
============================================================================================================ 1 failed in 0.64s =============================================================================================================
$ python -m mypy --config-file= --warn-unreachable --python-version 3.13 /tmp/a.py
/tmp/a.py:5: error: Implicit return in function which does not return [empty-body]
/tmp/a.py:9: error: Method must have at least one argument. Did you forget the "self" argument? [misc]
/tmp/a.py:11: error: Parameter 1 of Literal[...] is invalid [valid-type]
/tmp/a.py:18: error: Argument 1 to "assert_never" has incompatible type "E2"; expected "Never" [arg-type]
/tmp/a.py:23: error: Argument 1 to "assert_never" has incompatible type "<subclass of "member" and "E2">"; expected "Never" [arg-type]
Found 5 errors in 1 file (checked 1 source file)
Your Environment
- Mypy version used: #18675 head
- Mypy command-line flags:
--warn-unreachable --config-file= --python-version=3.13 - Mypy configuration options from
mypy.ini(and other config files): N/A - Python version used: 3.12
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the focused pytest case mypy/test/testcheck.py::TypeCheckSuite::check-enum.test and compare it with standalone mypy using the reported flags. Read the testDemo case in test-data/unit/check-enum.test and investigate the test fixtures and stubs involved. Done means the test and direct mypy invocation produce identical diagnostics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devtools, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100