pytest-dev / pytest-dev/pytest
[junitxml] logic for handling interlaced test reports in the pytest_runtest_logreport hook is xdist-specific but it could be xdist-agnostic (generic)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
Description
The src/_pytest/junitxml.py::pytest_runtest_logreport() hook implemented in the pytest built-in junitxml plugin has logic to handle interlaced test reports but that logic is too xdist-specific because it expects worker_id and item_index attributes in test reports that are set only by the pytest-xdist plugin. This part of code could be fully xdist-agnostic and depending only on uniqueness of report.nodeid attribute.
This is causing issues when generating JUnit XML report file with the pytest-cocotb plugin that allows to use pytest as regression manager for cocotb tests.
Captured stdout and stderr in report.capstdout and report.capstderr where report.when in ("setup", "call") are not included in JUnit XML report file when junit_logging is set to all|system-err|system-out.
I can prepare a Pull Request and refactor that hook. And add some unit tests.
Related issue: https://github.com/cocotb/cocotb/issues/5226
pip list
Package Version
-------------- -------------------
cocotb 2.1.0.dev0+e4b4975d
find-libpython 0.5.0
iniconfig 2.3.0
packaging 25.0
pluggy 1.6.0
pygments 2.19.2
pytest 9.0.2
OS
- Container image:
registry.gitlab.com/tymonx/ghdl:latest - Project: https://gitlab.com/tymonx/ghdl
Example
Git clone repository https://gitlab.com/tymonx/cocotb-junit-testing-ci-reports.git:
git clone https://gitlab.com/tymonx/cocotb-junit-testing-ci-reports.git
Change current working directory to it:
cd cocotb-junit-testing-ci-reports
Create environment using container image with GHDL simulator:
podman run -it --rm --workdir "$(pwd)" --volume "$(pwd):$(pwd):rw" --userns keep-id registry.gitlab.com/tymonx/ghdl:latest bash
Create Python virtual environment:
uv venv
Enter created Python virtual environment:
. .venv/bin/activate
Install Cocotb + Pytest:
uv pip install git+https://github.com/tymonx/cocotb.git@chore/xunit-reporter pytest
Run tests and generate JUnit XML report file junit.xml:
SIM=ghdl HDL_TOPLEVEL_LANG=vhdl WAVES=1 pytest -p cocotb_tools.pytest.plugin --override-ini=junit_logging=all --override-ini=junit_family=xunit1 --junit-xml=junit.xml
Generated JUnit XML report file junit.xml should contain COCOTB_RANDOM_SEED string in captured stderr:
grep COCOTB_RANDOM_SEED junit.xml
- a detailed description of the bug or problem you are having
- output of
pip listfrom the virtual environment you are using - pytest and operating system versions
- minimal example if possible
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/_pytest/junitxml.py at pytest_runtest_logreport() and read the report fields defined in src/_pytest/reports.py, especially nodeid, capstdout, and capstderr. Reproduce the issue with the provided cocotb command, then add unit tests covering interlaced reports and verify that the captured output appears in junit.xml for the requested junit_logging modes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 65/100