pytest-dev / pytest-dev/pytest
Teardown order mismatch with parametrized parent fixtures and scopes
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
Found during discussion of #11833 (I have reduced the size of the repro since I posted it in https://github.com/pytest-dev/pytest/pull/11833#issuecomment-1999818465)
import pytest
@pytest.fixture(scope="module", params=["a", "b"])
def fixture_1(request):
print("setup 1", request.param)
yield
print("\nteardown 1", request.param)
@pytest.fixture(scope="module")
def fixture_2():
print("setup 2")
yield
print("teardown 2")
def test_1(fixture_1, fixture_2): ...
output:
setup 1 a
setup 2
.
teardown 1 a
setup 1 b
.
teardown 1 b
teardown 2 # 2 is torn down last, perhaps surprisingly
If we reverse the order of the fixture parameters to test_1 we get an ordering where the stack is fully respected:
def test_1(fixture_2, fixture_1): ...
setup 2
setup 1 a
.
teardown 1 a
setup 1 b
.
teardown 1 b
teardown 2
What's happening in the first example is relatively straightforward -
test_1requestsfixture_2, which is set up and its finalizer is queued.test_1requestsfixture_1[a], which is set up and its finalizer is queued.- Test runs
- Because
fixture_1is parametrized, we run the test again. test_1requestsfixture_2- and since it has module scope the value is cached, so no setup is run (and since #11833 no finalizer is queued, not that it would make any difference here)test_1requestsfixture_1[b], it has a different cache key, sofixture_1[a]is torn down, andfixture_1[b]is setup with its finalizer queued.- Test finishes, and module is done, so we run the finalizers in reverse order they were added - which means
fixture_1[b]first, thenfixture_2, (and finallyfixture_1[a]is attempted as its still on the stack, but thats also irrelevant here).
I'm not even sure this is a bug per se, but I could at least see it being surprising to users as the documentation does not imply that the stack order is ever disrespected:
https://docs.pytest.org/en/8.0.x/how-to/fixtures.html#note-on-finalizer-order
Finalizers are executed in a first-in-last-out order. For yield fixtures, the first teardown code to run is from the right-most fixture, i.e. the last test parameter.
https://docs.pytest.org/en/8.0.x/how-to/fixtures.html#yield-fixtures-recommended
Once the test is finished, pytest will go back down the list of fixtures, but in the reverse order, taking each one that yielded, and running the code inside it that was after the yield statement.
Possible fixes:
- When interpreting the fixture parameters, magically reorder fixtures such that stack teardown ordering is not broken - in our example turning the first example into the second.
- This would probably be quite disruptive, and sounds very hard to code, so I don't think it's an option.
- Emit a warning when teardown ordering is possibly violated. Not super trivial to code, but when it's only controlling whether a warning is shown or not it needn't be perfect.
- Mention in the documentation that teardown order is not guaranteed to be the reverse, in the case of larger-scoped fixtures + parametrization.
- Also tear down
fixture_2whenfixture_1[a]is torn down.
I don't think this is a huge problem though, as I think in most practical cases an end user can work around this issue by changing fixture order, make fixture_2 depend on fixture_1, change scoping, etc, which makes me favor option 3 and possibly also 2.
Related to #4871
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the reduced Python fixture reproduction in the issue and reading the linked fixture-order documentation. Review discussion in #11833 and related issue #4871 before determining whether the resolution is a warning, documentation clarification, or teardown behavior change. Done requires an agreed resolution and corresponding tests or documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100