pytest-dev / pytest-dev/pytest
`@pytest.mark.*` priority seems wrong
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
Originally asked about in #10391 and deemed to be a bug by @RonnyPfannschmidt
Suppose I have a test class for which I want to turn warnings into errors. However, some of the parametrizations I'm going to test will emit warnings and I just want to ignore them. I thought, I can write something like
import pytest
import warnings
@pytest.mark.filterwarnings("error")
class TestMarkClass:
@pytest.mark.parametrize("_", [pytest.param(None, marks=pytest.mark.filterwarnings("ignore"))])
def test_foo(self, _):
warnings.warn("foo")
However, it seems that the class wide mark takes priority over the one in the parametrization. On the flip side, if I mark the individual tests like
class TestMarkMethods:
@pytest.mark.filterwarnings("error")
@pytest.mark.parametrize("_", [pytest.param(None, marks=pytest.mark.filterwarnings("ignore"))])
def test_bar(self, _):
warnings.warn("bar")
@pytest.mark.parametrize("_", [pytest.param(None, marks=pytest.mark.filterwarnings("ignore"))])
@pytest.mark.filterwarnings("error")
def test_baz(self, _):
warnings.warn("baz")
the mark in the parametrization takes priority. Running everything together with -rA set gives
======================================= FAILURES =======================================
_____________________________ TestMarkClass.test_foo[None] _____________________________
Traceback (most recent call last):
File "/home/user/main.py", line 9, in test_foo
warnings.warn("foo")
UserWarning: foo
======================================== PASSES ========================================
=============================== short test summary info ================================
PASSED main.py::TestMarkMethods::test_bar[None]
PASSED main.py::TestMarkMethods::test_baz[None]
FAILED main.py::TestMarkClass::test_foo[None] - UserWarning: foo
============================= 1 failed, 2 passed in 0.68s ==============================
This order seems wrong to me. I would expect the marker in the parametrization takes the highest priority, followed by the marker on the test and lastly the marker on the class.
$ pytest --version
pytest 7.1.3
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the reported class-level and test-level examples with pytest and inspect how pytest.mark.filterwarnings, pytest.param marks, and parametrization marks are collected and ordered. Add or update regression coverage for the three examples so parametrization marks take priority over test marks, which take priority over class marks, then verify the warning outcomes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100