pytest-dev / pytest-dev/pytest
Unexpected behavior of skipping entire module when calling unittest.skip("a")("b")
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
If aliasing unittest.skip with a value and inadvertently trying to pass an argument to the aliased copy, pytest will skip the entire module while unittest will throw an error.
import unittest
broken = unittest.skip("broken")
class MyTests(unittest.TestCase):
@broken
def test_good(self):
assert False
@broken("bar")
def test_evil(self):
assert False
pytest output
=============================================== test session starts ===============================================
platform darwin -- Python 3.11.2, pytest-7.2.2, pluggy-1.0.0
rootdir: ...
collected 0 items / 1 skipped
=============================================== 1 skipped in 0.01s ================================================
While running the same module via python -m unittest throws a unittest.case.SkipTest: broken error and fails loudly.
While the pattern of using the alias with a standard reason isn't a great pattern itself, nonetheless the behavior when it's used incorrectly is unprotected and easy to miss when you are trying to stable a test suite.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reproducer in the issue and compare pytest's handling of unittest.TestCase methods decorated with unittest.skip to python -m unittest. Trace pytest's collection and skip handling for the aliased decorator misuse; done means the incorrect call no longer silently skips the entire module and is covered by a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100