pytest-dev / pytest-dev/pytest

`@pytest.mark.*` priority seems wrong

Open
#10,406 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic: marks type: bug
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

Originally asked about in #10391 and deemed to be a bug by @RonnyPfannschmidt


Suppose I have a test class for which I want to turn warnings into errors. However, some of the parametrizations I'm going to test will emit warnings and I just want to ignore them. I thought, I can write something like

import pytest
import warnings


@pytest.mark.filterwarnings("error")
class TestMarkClass:
    @pytest.mark.parametrize("_", [pytest.param(None, marks=pytest.mark.filterwarnings("ignore"))])
    def test_foo(self, _):
        warnings.warn("foo")

However, it seems that the class wide mark takes priority over the one in the parametrization. On the flip side, if I mark the individual tests like

class TestMarkMethods:
    @pytest.mark.filterwarnings("error")
    @pytest.mark.parametrize("_", [pytest.param(None, marks=pytest.mark.filterwarnings("ignore"))])
    def test_bar(self, _):
        warnings.warn("bar")

    @pytest.mark.parametrize("_", [pytest.param(None, marks=pytest.mark.filterwarnings("ignore"))])
    @pytest.mark.filterwarnings("error")
    def test_baz(self, _):
        warnings.warn("baz")

the mark in the parametrization takes priority. Running everything together with -rA set gives

======================================= FAILURES =======================================
_____________________________ TestMarkClass.test_foo[None] _____________________________
Traceback (most recent call last):
  File "/home/user/main.py", line 9, in test_foo
    warnings.warn("foo")
UserWarning: foo
======================================== PASSES ========================================
=============================== short test summary info ================================
PASSED main.py::TestMarkMethods::test_bar[None]
PASSED main.py::TestMarkMethods::test_baz[None]
FAILED main.py::TestMarkClass::test_foo[None] - UserWarning: foo
============================= 1 failed, 2 passed in 0.68s ==============================

This order seems wrong to me. I would expect the marker in the parametrization takes the highest priority, followed by the marker on the test and lastly the marker on the class.

$ pytest --version
pytest 7.1.3

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the reported class-level and test-level examples with pytest and inspect how pytest.mark.filterwarnings, pytest.param marks, and parametrization marks are collected and ordered. Add or update regression coverage for the three examples so parametrization marks take priority over test marks, which take priority over class marks, then verify the warning outcomes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.