python / python/mypy

`mypy` behavior differs when run on a file directly vs a test

Open
#18,721 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug topic-developer topic-enum
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

Likely due to some test stubs inconsistency, mypy behaviour in test deviates from results of running full mypy on a file.

This is a follow-up for #18675.

To Reproduce

Add the following test to check-enum.test:

[case testDemo]
# flags: --python-version 3.13 --warn-unreachable
from enum import Enum, member, nonmember
from typing import Literal, Never

def assert_never(_: Never) -> Never: ...

class E2(Enum):
    @member
    def C() -> None: ...  # E: Method must have at least one argument. Did you forget the "self" argument?  [misc]

c: Literal[E2.C]  # E: Parameter 1 of Literal[...] is invalid  [valid-type]

def check_2(e: E2) -> None:
    match e:
        case E2.C:
            pass
        case other:
            assert_never(other)  # E: Argument 1 to "assert_never" has incompatible type "E2"; expected "Never"  [arg-type]

    if e is E2.C:
        pass
    else:
        assert_never(e)  # E: Argument 1 to "assert_never" has incompatible type "<subclass of "enum.member[Callable[[], None]]" and "__main__.E2">"; expected "Never"  [arg-type]
[builtins fixtures/enum.pyi]

Now copy the test code to a separate file and run mypy on it.

Expected Behavior

Test and mypy outputs are identical.

Actual Behavior

$ pytest mypy/test/testcheck.py::TypeCheckSuite::check-enum.test -k testDemo 
=========================================================================================================== test session starts ============================================================================================================
platform linux -- Python 3.12.6, pytest-8.3.3, pluggy-1.5.0
rootdir: /home/stas/Documents/Work/mypy
configfile: pyproject.toml
plugins: cov-5.0.0, xdist-3.6.1
2 workers [1 item]      
F                                                                                                                                                                                                                                    [100%]
================================================================================================================= FAILURES =================================================================================================================
_________________________________________________________________________________________________________________ testDemo _________________________________________________________________________________________________________________
[gw0] linux -- Python 3.12.6 /home/stas/Documents/Work/mypy/.venv12/bin/python3
data: /home/stas/Documents/Work/mypy/test-data/unit/check-enum.test:2516:
Failed: Unexpected type checker output (/home/stas/Documents/Work/mypy/test-data/unit/check-enum.test, line 2516)
----------------------------------------------------------------------------------------------------------- Captured stderr call -----------------------------------------------------------------------------------------------------------
Expected:
  main:9: error: Method must have at least one argument. Did you forget the "self" argument?  [misc] (diff)
  main:11: error: Parameter 1 of Literal[...] is invalid  [valid-type] (diff)
  main:18: error: Argument 1 to "assert_never" has incompatible type "E2"; expected "Never"  [arg-type] (diff)
  main:23: error: Argument 1 to "assert_never" has incompatible type "<subclass of "enum.member[Callable[[], None]]" and "__main__.E2">"; expected "Never"  [arg-type] (diff)
Actual:
  main:9: error: Method must have at least one argument. Did you forget the "self" argument? (diff)
  main:11: error: Parameter 1 of Literal[...] is invalid (diff)
  main:18: error: Argument 1 to "assert_never" has incompatible type "Union[member[Callable[[], None]], E2]"; expected "Never" (diff)

Alignment of first line difference:
  E: ...the "self" argument?  [misc]
  A: ...the "self" argument?
                            ^
Update the test output using --update-data (implies -n0; you can additionally use the -k selector to update only specific tests)
========================================================================================================= short test summary info ==========================================================================================================
FAILED mypy/test/testcheck.py::TypeCheckSuite::check-enum.test::testDemo
============================================================================================================ 1 failed in 0.64s =============================================================================================================

$ python -m mypy --config-file= --warn-unreachable  --python-version 3.13 /tmp/a.py

/tmp/a.py:5: error: Implicit return in function which does not return  [empty-body]
/tmp/a.py:9: error: Method must have at least one argument. Did you forget the "self" argument?  [misc]
/tmp/a.py:11: error: Parameter 1 of Literal[...] is invalid  [valid-type]
/tmp/a.py:18: error: Argument 1 to "assert_never" has incompatible type "E2"; expected "Never"  [arg-type]
/tmp/a.py:23: error: Argument 1 to "assert_never" has incompatible type "<subclass of "member" and "E2">"; expected "Never"  [arg-type]
Found 5 errors in 1 file (checked 1 source file)

Your Environment

  • Mypy version used: #18675 head
  • Mypy command-line flags: --warn-unreachable --config-file= --python-version=3.13
  • Mypy configuration options from mypy.ini (and other config files): N/A
  • Python version used: 3.12

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the focused pytest case mypy/test/testcheck.py::TypeCheckSuite::check-enum.test and compare it with standalone mypy using the reported flags. Read the testDemo case in test-data/unit/check-enum.test and investigate the test fixtures and stubs involved. Done means the test and direct mypy invocation produce identical diagnostics.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.