pytest-dev / pytest-dev/pytest
Fixture order considering global states
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
Hi,
This section of the documentation automatic grouping of tests by fixtures instances almost solves one need we are having, related to fixtures dealing with global states.
Problem example
To exemplify, let's simplify the example on the documentation, with seemly independent tests, each one with its fixture (modarg with test_1; and otherarg with test_2).
modarg and otherarg change the same global state.
# content of test_module.py
import pytest
@pytest.fixture(scope="module", params=['a', 'b'])
def modarg(request):
return request.param
@pytest.fixture(scope="module", params=['a', 'b'])
def otherarg(request):
return request.param
def test_0(otherarg):
print(" RUN test0 with otherarg %s" % otherarg)
def test_1(modarg):
print(" RUN test1 with modarg %s" % modarg)
It is not a problem because pytest orders in such a way that all tests using one fixture are executed, then all tests using the another:
test_example2.py::test_0[a] PASSED
test_example2.py::test_0[b] PASSED
test_example2.py::test_1[a] PASSED
test_example2.py::test_1[b] PASSED
Now, let's say I want to refactor the ['a', 'b'] parameters into a common fixture.
The problem now is that pytest tries to first use the new, highest level uppermost fixture first, now interleaving the execution of the fixtures modarg and otherarg (which is a problem because they are module scoped and are dealing with the same shared resource).
# content of test_module.py
import pytest
@pytest.fixture(scope="module", params=['a', 'b'])
def uppermost(request):
return request.param
@pytest.fixture(scope="module")
def modarg(uppermost):
return uppermost
@pytest.fixture(scope="module")
def otherarg(uppermost):
return uppermost
def test_0(otherarg):
print(" RUN test0 with otddherarg %s" % otherarg)
def test_1(modarg):
print(" RUN test1 with modarg %s" % modarg)
test_example.py::test_0[a] PASSED
test_example.py::test_1[a] PASSED
test_example.py::test_0[b] PASSED
test_example.py::test_1[b] PASSED
Needed behaviour
In our situation, the resource is a piece of hardware, and the module-scoped fixture is an expensive set-up of that hardware (start-up and configuration).
For this case, it would solve to be able to specify that a given fixture share a resource - so, when ordering, pytest can make sure that there is no interleaving between the execution of tests that share that same resource.
So in our example, we would possibly have this:
@pytest.fixture(scope="module", shared_resource="myhardware")
def modarg(uppermost):
return uppermost
@pytest.fixture(scope="module", shared_resource="myhardware")
def otherarg(uppermost):
return uppermost
And that's it. I would like to hear if this is sensible to add or not, probably more experienced developers might spot any detrimental side effects of this. I would be willing to implement this myself once we decide it could and should be done.
Thanks!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked documentation section on automatic grouping of tests by fixture instances and reproduce the two ordering examples from the issue. Then inspect pytest's fixture ordering and parametrization behavior to determine where shared-resource semantics would belong. Done means a decided design, implementation, and tests showing that fixtures sharing a resource are not interleaved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100