pytest-dev / pytest-dev/pytest

`--last-failed` does not (always?) see tests which failed in a subtest

Open
#14,575 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

When running a unittest-based test suite which makes use of subtests, failed subtests are correctly reported at the end of a run (possibly a bit too much as it's hitting #13905 / #13986), however when re-running tests using --lf they seem to skipped.

Originally observed on pytest 9.0.2 / uv python 3.12, reproduced on 9.0.3 / uv python 3.14, running on linux mint 22.3

$ uv run pytest
==== test session starts ===
platform linux -- Python 3.14.2, pytest-9.0.3, pluggy-1.6.0
rootdir: /tmp/testdir
configfile: pytest.ini
collected 1 item                                                                                                                                                                                                                             

test_qux.py .                                                                                                                                                                                                                        [100%]

=== FAILURES ===
__________________________________________________________________________________________________________ FooTest.testX (index=1) ___________________________________________________________________________________________________________

self = <test_qux.FooTest testMethod=testX>

    def testX(self):
        for i in range(5):
            with self.subTest(index=i):
>               self.assertEqual(i % 2, 0)
E               AssertionError: 1 != 0

test_qux.py:9: AssertionError
__________________________________________________________________________________________________________ FooTest.testX (index=3) ___________________________________________________________________________________________________________

self = <test_qux.FooTest testMethod=testX>

    def testX(self):
        for i in range(5):
            with self.subTest(index=i):
>               self.assertEqual(i % 2, 0)
E               AssertionError: 1 != 0

test_qux.py:9: AssertionError
======================================================================================================= short test summary info ===
SUBFAILED(index=1) test_qux.py::FooTest::testX - AssertionError: 1 != 0
SUBFAILED(index=3) test_qux.py::FooTest::testX - AssertionError: 1 != 0
=== 2 failed, 1 passed in 0.04s ===
$ uv run pytest --cache-show 
=== test session starts ===
platform linux -- Python 3.14.2, pytest-9.0.3, pluggy-1.6.0
rootdir: /tmp/testdir
configfile: pytest.ini
cachedir: /tmp/testdir/.pytest_cache
--- cache values for '*' ---
cache/nodeids contains:
  ['test_qux.py::FooTest::testX']

=== no tests ran in 0.00s ===
$ uv run pytest --lf --lfnf=none
=== test session starts ===
platform linux -- Python 3.14.2, pytest-9.0.3, pluggy-1.6.0
rootdir: /tmp/testdir
configfile: pytest.ini
collected 1 item / 1 deselected / 0 selected                                                                                                                                                                                                 
run-last-failure: no previously failed tests, deselecting all items.

=== 1 deselected in 0.00s ===
# test_qux.py
import unittest


class FooTest(unittest.TestCase):
    def testX(self):
        for i in range(5):
            with self.subTest(index=i):
                self.assertEqual(i % 2, 0)
# pyproject.toml (mostly so uv knows to fetch pytest)
[project]
name = "testdir"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.14"
dependencies = []

[dependency-groups]
dev = [
    "pytest>=9.0.3",
]

And an empty pytest.ini to ensure pytest gets the correct rootdir (not sure it's necessary with the pyproject.toml but...)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reproducer in test_qux.py and run the shown pytest, --cache-show, and --lf commands using the pyproject.toml setup. Trace how unittest subTest failures are reported and stored for --last-failed; done means the failed subtests are recorded and selected on the next run without breaking ordinary failure handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.