pytest-dev / pytest-dev/pytest
Dynamically added xfail mark does not work on fixtures preparations phase
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
Normally, decoration with xfail prevents from failure due to ERROR's in loading fixtures. But sometimes decorations are not possible as they cannot load functions or lazy-load other fixtures or configoptions. See also https://github.com/pytest-dev/pytest/issues/7395
Dynamically applied marks should do the same e.i. resolves to XFAIL on running pytest versus `test_xfail instead of ERROR.
@pytest.fixture
def conditional_fxt():
return "a"
@pytest.fixture
def mark_xfail_conditionally(request, conditional_fxt):
if conditional_fxt == "a":
request.node.add_marker(pytest.mark.xfail(reason="test"))
@pytest.fixture
def setup(mark_xfail_conditionally):
raise RuntimeError()
def test_xfail(setup):
...
also didn't not work with request.applymarker (IDK the usecases BTW).
Workarounds that are not always possible/desirable:
- using decoration
- using
pytest.xfailinstead of marks (won't run the further code) - making setup fixture callable
@pytest.fixture
def setup(mark_xfail_conditionally):
def fn():
raise RuntimeError()
return fn
def test_xfail(setup):
setup()
...
Checked on:
- pytest 6.2.2, pytest 7.2.2
- ubuntu
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the fixture preparation phase and the handling of dynamically added xfail marks, using the provided conditional_fxt, mark_xfail_conditionally, setup, and test_xfail reproduction. Compare dynamically applied marks with decoration and request.applymarker behavior. Done means a fixture-preparation RuntimeError is reported as XFAIL rather than ERROR when the xfail mark is added dynamically.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100