pytest-dev / pytest-dev/pytest
metafunc.parametrize(..., scope="session") fails when mixed with other parametrizations
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
Both, python 2.7.13 and 3.5.3, pytest 3.6.1, no additional pytest plugins.
# conftest.py
import pytest
KNOWN_BRANCHES = {
# The number means how many times given branch has been set-up
"branch_1": 0,
"branch_2": 0,
}
def pytest_generate_tests(metafunc):
if 'tested_branch' in metafunc.fixturenames:
branches = KNOWN_BRANCHES.keys()
metafunc.parametrize("tested_branch", branches, scope="session")
@pytest.fixture(scope="session")
def repo_preparation(tested_branch):
""" If it would have a "session" scopoe then the value should be always 1 for each branch."""
KNOWN_BRANCHES[tested_branch] += 1
return KNOWN_BRANCHES[tested_branch]
And the simplest reproduction:
# test_parametrization.py
import pytest
def test_nothing(repo_preparation):
assert repo_preparation == 1 # that passes
@pytest.mark.parametrize("_", ["param_1", "param_2"])
def test_basic_parametrization(repo_preparation, _):
# that causes parametrization confisuon
assert repo_preparation == 1, "Repo prepared more than once."
Gives such an result:
$ pytest -v test_parametrization.py
================================== test session starts ==================================
platform linux -- Python 3.5.3, pytest-3.6.1, py-1.5.3, pluggy-0.6.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /home/kamichal/ecws/pyplayground, inifile:
collected 6 items
test_parametrization.py::test_nothing[branch_1] PASSED [ 16%]
test_parametrization.py::test_basic_parametrization[branch_1-param_1] PASSED [ 33%]
test_parametrization.py::test_nothing[branch_2] PASSED [ 50%]
test_parametrization.py::test_basic_parametrization[branch_1-param_2] PASSED [ 66%]
test_parametrization.py::test_basic_parametrization[branch_2-param_1] FAILED [ 83%]
test_parametrization.py::test_basic_parametrization[branch_2-param_2] FAILED [100%]
======================================= FAILURES ========================================
_____________________ test_basic_parametrization[branch_2-param_1] ______________________
repo_preparation = 2, _ = 'param_1'
@pytest.mark.parametrize("_", ["param_1", "param_2"])
def test_basic_parametrization(repo_preparation, _):
# that causes parametrization confisuon
> assert repo_preparation == 1, "Repo prepared more than once."
E AssertionError: Repo prepared more than once.
E assert 2 == 1
test_parametrization.py:12: AssertionError
_____________________ test_basic_parametrization[branch_2-param_2] ______________________
repo_preparation = 3, _ = 'param_2'
@pytest.mark.parametrize("_", ["param_1", "param_2"])
def test_basic_parametrization(repo_preparation, _):
# that causes parametrization confisuon
> assert repo_preparation == 1, "Repo prepared more than once."
E AssertionError: Repo prepared more than once.
E assert 3 == 1
test_parametrization.py:12: AssertionError
========================== 2 failed, 4 passed in 0.03 seconds ===========================
Setting indirect=True in the metafunc.parametrize call fixes that problem, but there is no information that it's required for larger scopes to work properly. There is only an advice to do so, but...
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure using conftest.py and test_parametrization.py with pytest 3.6.1, comparing session-scoped parametrization with and without indirect=True. Trace pytest's parametrization and fixture setup behavior, then add a regression test or documentation clarification so mixed parametrizations with session scope behave as intended.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100