pytest-dev / pytest-dev/pytest

All tests are run when only last failed tests are requested

Open
#13,614 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

topic: collection
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

Observed behavior:

When I run pytest -s --lf --lfnf=none tests and all the tests that failed during the previous run of the test suite where removed, all the tests are run.

Expected behavior:

I would expect that no tests are run at all in the above-mentioned situation, because the user explicitly requested to only run the tests that failed previously. Given that there are none, no tests should run.

Minimal reproduction:

Step 1: Run pytest tests with the following tests/test_test.py file:

def test_test1():
    raise Exception()
 
def test_test2():
    return

This produces the following output:

Test session starts (platform: linux, Python 3.12.11, pytest 8.4.1, pytest-sugar 1.0.0)
Using --randomly-seed=533107519
rootdir: /tmp/test
configfile: pyproject.toml
plugins: time-machine-2.16.0, env-1.1.5, cov-6.2.1, instafail-0.5.0, sugar-1.0.0, timeout-2.4.0, randomly-3.16.0, postgresql-7.0.2, asyncio-1.1.0, logfire-4.0.0
asyncio: mode=Mode.AUTO, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 2 items                                                                                                                                                     


――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― test_test1 ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

    def test_test1():
>       raise Exception()
E       Exception

tests/test_test.py:2: Exception

 tests/test_test.py ⨯✓                                                                                                                                  100% ██████████
======================================================================= short test summary info =======================================================================
FAILED tests/test_test.py::test_test1 - Exception

Results (0.06s):
       1 passed
       1 failed
         - tests/test_test.py:1 test_test1

Step 2: Run pytest --lf --lfnf=none tests with the following tests/test_test.py file (remove test1):

def test_test2():
    return

This produces the following output:

Test session starts (platform: linux, Python 3.12.11, pytest 8.4.1, pytest-sugar 1.0.0)
Using --randomly-seed=304109397
rootdir: /tmp/test
configfile: pyproject.toml
plugins: time-machine-2.16.0, env-1.1.5, cov-6.2.1, instafail-0.5.0, sugar-1.0.0, timeout-2.4.0, randomly-3.16.0, postgresql-7.0.2, asyncio-1.1.0, logfire-4.0.0
asyncio: mode=Mode.AUTO, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 1 item                                                                                                                                                      
run-last-failure: 1 known failures not in selected tests

 tests/test_test.py ✓                                                                                                                                   100% ██████████

Results (0.02s):
       1 passed

Software versions:

  • The above-mentioned issue happens on pytest==8.4.1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the two-step case with tests/test_test.py using pytest tests and then pytest --lf --lfnf=none tests. Start from the run-last-failure behavior shown in the output; done means the second command selects and runs no tests when its previous failures are absent.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.