pytest-dev / pytest-dev/pytest

Surprising behavior when passing class/function to marker

Open
#8,499 8 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

topic: marks
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

Consider the following example:

import pytest

class Context:  # actually imported from the module under test

    def __init__(self, config):
        pass

@pytest.mark.use_context(Context)
def test_with_special_context():
    pass

(and imagine a fixture then using the passed context, though that's not so relevant here)

This results in:

============================= test session starts ==============================
platform linux -- Python 3.9.2, pytest-6.3.0.dev324+gca4a7a085, py-1.10.0, pluggy-0.13.1
rootdir: /home/florian/tmp/rep, configfile: pytest.ini
collected 0 items                                                              

============================ no tests ran in 0.00s =============================

which is surprising, isn't it? When I first saw this, I had no idea what was going on (in my case, there were other tests, and when I introduced the marker, tests suddenly started disappearing).

While writing a minimal reproducer, I discovered what's going on when removing the __init__ for Context:

__________________________ ERROR collecting test_x.py __________________________
test_x.py:7: in <module>
    def test_with_special_context():
E   TypeError: Context() takes no arguments

Still weird, albeit slightly less so. This is happening because the @pytest.mark decorator is written in a way it supports being called as both @pytest.mark.mymark and @pytest.mark.mymark(...):

https://github.com/pytest-dev/pytest/blob/ca4a7a085e853636e400ac68f38609e5ed5e8b3d/src/_pytest/mark/structures.py#L360-L368

However, the Context argument I passed to the marker just happens to be a class, so pytest incorrectly assumes that's the test class to decorate, helpfully replacing my original test function with the Context class...

I have no idea how to fix this though, while still correctly supporting our "hybrid" decorators.

I guess a slight improvement would be for pytest to check whether the argument is a test class/function (i.e. follows the naming conventions for it to be collected as test, and also in a test module rather than a different module). That of course makes this issue even more subtle when it appears, but at least it'd solve this particular case.

I really wish Python decorators somehow worked in a less confusing way though...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the hybrid decorator logic in src/_pytest/mark/structures.py at the linked lines, then reproduce the behavior with the Context example from the issue. Done means passing a class or function as marker data no longer replaces the decorated test or causes collection to fail, while the existing hybrid decorator forms continue to work.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.