pytest-dev / pytest-dev/pytest
Please make it possible to "mark" subtests as `xfail`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
What's the problem this feature will solve?
Currently, it isn't possible to conveniently mark a subtest as XFAIL-ing with full support for xfail_strict=True.
Describe the solution you'd like
Ideally, I'd like to be able to do something like:
def test_foo(subtests):
with subtests.test(msg=..., xfail=condition):
...
that would mark this one subtest as XFAIL, the same way pytest.mark.xfail marks the whole test as XFAIL-ing.
Alternative Solutions
- Marking the whole test with
pytest.mark.xfailcauses it to unconditionally fail withxfail_strict=True. Reading the comments in the code, I infer that it isn't supported. And even if it were, more precision would be desirable.$ pytest test.py -v -o xfail_strict=True ========================================================= test session starts ========================================================= platform linux -- Python 3.12.12, pytest-9.1.0.dev148+g0e9db5fa5, pluggy-1.6.0 -- /tmp/foo/.venv/bin/python3 cachedir: .pytest_cache rootdir: /tmp/foo collected 1 item test.py::test_foo SUBXFAIL[xfail] (foo) [100%] test.py::test_foo FAILED [100%] ============================================================== FAILURES =============================================================== ______________________________________________________________ test_foo _______________________________________________________________ [XPASS(strict)] foo ======================================================= short test summary info ======================================================= FAILED test.py::test_foo - [XPASS(strict)] foo ==================================================== 1 failed, 1 xfailed in 0.01s ===================================================== - It is possible to call
pytest.xfail()directly from the test. However, this requires a bit of handiwork to "reimplement"xfail_strict.
Additional context
n/a
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing how subtests.test, pytest.mark.xfail, and pytest.xfail interact with xfail_strict=True. Compare the existing subtest reporting and strict-XFAIL behavior, then verify that the proposed per-subtest xfail condition reports expected failures and strict unexpected passes without affecting other subtests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100