python / python/mypy

[dev] wrong-version version tests disappear completely, but should probably skip instead

Open
#20,398 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug topic-tests
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Consider the following:

uv run runtests.py testTemplateString
usage: runtests.py [self] [self-packaging] [lint] [pytest-fast] [pytest-cmdline] [pytest-slow] [mypyc-fast] [pytest-extra] [mypyc-extra] [names of individual tests and files...]

Run the given tests. If given no arguments, run everything except pytest-extra and mypyc-extra. Unrecognized arguments will be interpreted as individual test names / substring expressions (or, if they end in .test, individual test files) and this script will try to run them.
run testTemplateString: ['pytest', '-n0', '-k', 'testTemplateString']
========================================================================================================== test session starts ===========================================================================================================
platform win32 -- Python 3.13.7, pytest-8.4.2, pluggy-1.6.0
rootdir: C:\Users\wyatt\files\gits\wyattscarpenter\cfork\mypy
configfile: pyproject.toml
testpaths: mypy/test, mypyc/test
plugins: cov-7.0.0, xdist-3.8.0
collected 13431 items / 13431 deselected / 0 selected

======================================================================================================= 13431 deselected in 1.78s ========================================================================================================

FAILED: testTemplateString

This is a real test, in parse-python314.test (and also, oddly enough, in check-python314.test — not sure what's up with that) but, apparently, because my python version is <3.14, it disappears. It should probably pytest skip instead (yellow "s" on the list of results), that seems like the correct result semantically, and is less mysterious than the test just not being there.

A good place to start investigating this change might be /mypy/test/testparse.py.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in /mypy/test/testparse.py and inspect how parse-python314.test and check-python314.test are selected when running testTemplateString on Python versions below 3.14. Run the provided runtests.py command to reproduce the zero-selected result. Done means the version-inapplicable test is reported as skipped rather than disappearing, with the relevant tests still behaving correctly on supported versions.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.