pytest-dev / pytest-dev/pytest
`--last-failed` does not (always?) see tests which failed in a subtest
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
When running a unittest-based test suite which makes use of subtests, failed subtests are correctly reported at the end of a run (possibly a bit too much as it's hitting #13905 / #13986), however when re-running tests using --lf they seem to skipped.
Originally observed on pytest 9.0.2 / uv python 3.12, reproduced on 9.0.3 / uv python 3.14, running on linux mint 22.3
$ uv run pytest
==== test session starts ===
platform linux -- Python 3.14.2, pytest-9.0.3, pluggy-1.6.0
rootdir: /tmp/testdir
configfile: pytest.ini
collected 1 item
test_qux.py . [100%]
=== FAILURES ===
__________________________________________________________________________________________________________ FooTest.testX (index=1) ___________________________________________________________________________________________________________
self = <test_qux.FooTest testMethod=testX>
def testX(self):
for i in range(5):
with self.subTest(index=i):
> self.assertEqual(i % 2, 0)
E AssertionError: 1 != 0
test_qux.py:9: AssertionError
__________________________________________________________________________________________________________ FooTest.testX (index=3) ___________________________________________________________________________________________________________
self = <test_qux.FooTest testMethod=testX>
def testX(self):
for i in range(5):
with self.subTest(index=i):
> self.assertEqual(i % 2, 0)
E AssertionError: 1 != 0
test_qux.py:9: AssertionError
======================================================================================================= short test summary info ===
SUBFAILED(index=1) test_qux.py::FooTest::testX - AssertionError: 1 != 0
SUBFAILED(index=3) test_qux.py::FooTest::testX - AssertionError: 1 != 0
=== 2 failed, 1 passed in 0.04s ===
$ uv run pytest --cache-show
=== test session starts ===
platform linux -- Python 3.14.2, pytest-9.0.3, pluggy-1.6.0
rootdir: /tmp/testdir
configfile: pytest.ini
cachedir: /tmp/testdir/.pytest_cache
--- cache values for '*' ---
cache/nodeids contains:
['test_qux.py::FooTest::testX']
=== no tests ran in 0.00s ===
$ uv run pytest --lf --lfnf=none
=== test session starts ===
platform linux -- Python 3.14.2, pytest-9.0.3, pluggy-1.6.0
rootdir: /tmp/testdir
configfile: pytest.ini
collected 1 item / 1 deselected / 0 selected
run-last-failure: no previously failed tests, deselecting all items.
=== 1 deselected in 0.00s ===
# test_qux.py
import unittest
class FooTest(unittest.TestCase):
def testX(self):
for i in range(5):
with self.subTest(index=i):
self.assertEqual(i % 2, 0)
# pyproject.toml (mostly so uv knows to fetch pytest)
[project]
name = "testdir"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.14"
dependencies = []
[dependency-groups]
dev = [
"pytest>=9.0.3",
]
And an empty pytest.ini to ensure pytest gets the correct rootdir (not sure it's necessary with the pyproject.toml but...)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reproducer in test_qux.py and run the shown pytest, --cache-show, and --lf commands using the pyproject.toml setup. Trace how unittest subTest failures are reported and stored for --last-failed; done means the failed subtests are recorded and selected on the next run without breaking ordinary failure handling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100