pytest-dev / pytest-dev/pytest
No skipped message in JUnit report for unexpectedly passed tests
Open
Nobody has claimed this yet.
plugin: junitxml
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
In 2.9.2 version of pytest for unexpectedly passed tests there was skipped tag with message in JUnit XML report like this:
<testsuite errors="0" failures="0" name="pytest" skips="1" tests="1" time="6.368">
<testcase classname="test_example" file="test_example.py" line="127" name="test_example" time="0.22827982902526855">
<skipped message="xfail-marked test passes unexpectedly">None</skipped>
<system-out>
</system-out>
</testcase>
</testsuite>
But since version 3.0.0 skipped tag has gone:
<testsuite errors="0" failures="0" name="pytest" skips="1" tests="1" time="6.368">
<testcase classname="test_example" file="test_example.py" line="127" name="test_example" time="0.22827982902526855">
<system-out>
</system-out>
</testcase>
</testsuite>
So for now there is no way to distinguish passed and unexpectedly passed tests.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the example with pytest 2.9.2 and 3.0.0, focusing on an xfail-marked test that passes unexpectedly and its JUnit XML output. Compare the generated testcase elements; done when the report distinguishes unexpectedly passed tests with the skipped tag and message described in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100