pytest-dev / pytest-dev/pytest
self.skipTest does not produce the same results as unittest does
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
Given this test case:
from unittest import TestCase, main
class T(TestCase):
def test_foo(self):
for i in range(5):
with self.subTest(msg="custom", i=i):
if i % 2 == 0:
self.skipTest('even number')
if __name__ == '__main__':
main()
Running with python:
λ python .tmp\test-ut-skip.py
sss
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK (skipped=3)
Running with pytest:
λ pytest .tmp\test-ut-skip.py -q
ss [100%]
2 skipped in 0.01 seconds
The problem is that TestCaseFunction.addSkip will append the exception info to a list, but pytest_runtest_makereport will only pop one item off the list. We need to find a way to hook into TestCase.subTest to issue pytest_runtest_logreport from within the with statement.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the example with both unittest and pytest, then inspect TestCaseFunction.addSkip and pytest_runtest_makereport. Trace how each self.subTest skip is recorded and reported; done means pytest reports all three skipped subtests, matching unittest's result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100