pytest-dev / pytest-dev/pytest
junitxml: Incorrect (?) file reported for inherited tests
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
When a class A inherits tests from a class B in another file, the test cases are reported as belonging to the file that houses class B.
Example:
# example/base.py
class Base:
def test_something(self):
assert 1 == 1
# example/test_something.py
from unittest import TestCase
from .base import Base
class MyTest(Base, TestCase):
pass
> pytest -o junit_family=xunit1 --junit-xml=test.xml example/test_something.py
> cat test.xml
<?xml version="1.0" encoding="utf-8"?>
<testsuites>
<testsuite name="pytest" errors="0" failures="0" skipped="0" tests="1" time="0.039" timestamp="2021-12-07T09:54:49.284569" hostname="hello">
<testcase classname="example.test_something.MyTest" name="test_something" file="example/base.py" line="1" time="0.001" />
</testsuite>
</testsuites>
I'm not sure if this is correct or not. In CircleCI, it prevents the "split tests by timing" feature from working correctly because it cannot find e.g. example/test_something.py in the XML doc. Is this intended? Is there a better method for inheriting tests?
> pip freeze
astroid==2.9.0
attrs==21.2.0
iniconfig==1.1.1
isort==5.10.1
lazy-object-proxy==1.6.0
mccabe==0.6.1
packaging==21.3
platformdirs==2.4.0
pluggy==1.0.0
py==1.11.0
pyparsing==3.0.6
pytest==6.2.5
toml==0.10.2
typing-extensions==4.0.1
wrapt==1.13.3
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the inherited-test case using example/base.py and example/test_something.py with pytest's --junit-xml option, then inspect the junitxml reporting path that produces the testcase file attribute. Done means the XML reports the inherited test under the file that was executed, example/test_something.py, while preserving the testcase and classname details.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100