pytest-dev / pytest-dev/pytest

junitxml: Incorrect (?) file reported for inherited tests

Open
#9,388 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

plugin: junitxml topic: collection type: question
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

When a class A inherits tests from a class B in another file, the test cases are reported as belonging to the file that houses class B.

Example:

# example/base.py

class Base:
    def test_something(self):
        assert 1 == 1
# example/test_something.py
from unittest import TestCase

from .base import Base

class MyTest(Base, TestCase):
    pass
> pytest -o junit_family=xunit1 --junit-xml=test.xml example/test_something.py
> cat test.xml
<?xml version="1.0" encoding="utf-8"?>
<testsuites>
  <testsuite name="pytest" errors="0" failures="0" skipped="0" tests="1" time="0.039" timestamp="2021-12-07T09:54:49.284569" hostname="hello">
    <testcase classname="example.test_something.MyTest" name="test_something" file="example/base.py" line="1" time="0.001" />
  </testsuite>
</testsuites>

I'm not sure if this is correct or not. In CircleCI, it prevents the "split tests by timing" feature from working correctly because it cannot find e.g. example/test_something.py in the XML doc. Is this intended? Is there a better method for inheriting tests?

> pip freeze
astroid==2.9.0
attrs==21.2.0
iniconfig==1.1.1
isort==5.10.1
lazy-object-proxy==1.6.0
mccabe==0.6.1
packaging==21.3
platformdirs==2.4.0
pluggy==1.0.0
py==1.11.0
pyparsing==3.0.6
pytest==6.2.5
toml==0.10.2
typing-extensions==4.0.1
wrapt==1.13.3

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the inherited-test case using example/base.py and example/test_something.py with pytest's --junit-xml option, then inspect the junitxml reporting path that produces the testcase file attribute. Done means the XML reports the inherited test under the file that was executed, example/test_something.py, while preserving the testcase and classname details.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.