apache / apache/maven-surefire
[SUREFIRE-1644] Identify flaky tests in Surefire report
- Dominant language
- Java
- Stars
- 461
- Forks
- 588
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 19
Description
**[André Carmo](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=atmcarmo)** opened **[SUREFIRE-1644](https://issues.apache.org/jira/browse/SUREFIRE-1644?redirect=false)** and commented
Hi all,
There is a feature that allows to rerun failing tests: https://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html
This feature outputs to the surefire XML report some elements like flakyFailure, flakyError, and others.
However, when generating the surefire report (invoking something like mvn surefire-report:report-only) those XML elements are not used and are not considered for the HTML report. That would be a great improvement.
I can try to make a pull request for that, but please give me an overview first on where to look at to try to cut some time and some effort.
---
**Issue Links:**
- [SUREFIRE-1751](https://issues.apache.org/jira/browse/SUREFIRE-1751) Surefire report shows flaky tests as failures
- [SUREFIRE-1345](https://issues.apache.org/jira/browse/SUREFIRE-1345) Support flakyFailure and flakyError in TestSuiteXmlParser
(_**"depends upon"**_)
Contributor guide
Research direction
Start with the surefire-report:report-only flow and the TestSuiteXmlParser mentioned in SUREFIRE-1345. Trace how flakyFailure and flakyError in the Surefire XML are handled when producing the HTML report, then verify that flaky tests are represented correctly rather than as ordinary failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100