box / box/flaky

Flaky reruns autouse session-scoped fixtures when a tests fails one of its runs

Open
#207 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
398
Forks
57
PR merge metrics
No merged PRs in 30d

Description

When a test fails and is rerun, Flaky reruns session-scoped autouse fixtures. For example, running this test:

```
import pytest

@pytest.fixture(scope="session", autouse=True)
def my_session_scoped_autouse_fixture():
print("\nIn session scope autouse fixture")

@pytest.mark.flaky(max_runs=3)
def test_flaky_reruns(local_list = []):
local_list.append(0)
assert len(local_list)==3
```

calls the fixture 3 times:

```
============================= test session starts ==============================
collecting ... collected 1 item

test_it.py::test_flaky_reruns
In session scope autouse fixture

In session scope autouse fixture

In session scope autouse fixture
PASSED

============================== 1 passed in 0.01s ===============================

Process finished with exit code 0
```
This can cause unexpected behaviors. In my case, I'm using the pytest-playwright package which has a session-scoped autouse fixture that deletes the output directory where traces are stored for failing tests. If a test fails and is rerun, any previously created trace files are deleted.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.