jenkinsci / jenkinsci/junit-plugin
Support <flakyFailure> and <flakyError> in junit xml report
- Dominant language
- Java
- Stars
- 87
- Forks
- 351
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 2
Description
Both Maven and Gradle support automatically re-running tests on failure and marking them as flaky if they pass on the retry. For better or worse, flaky tests are a common problem for integration and system tests. Marking such tests as flaky would be very helpful.
The [Flaky Test Handler](https://plugins.jenkins.io/flaky-test-handler/) plugin has this functionality and more, but it was last released 5 years ago and not actively maintained. It would be great if the basic reporting functionality was part of the core junit plugin.
Relevant documentation for Maven surefire:
> flakyFailure and flakyError elements will be used in the generated xml report to include information of each failing re-runs.
https://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html
Similarly, Gradle:
> When mergeReruns is enabled, if a test fails but is then retried and succeeds, its failures will be recorded as \ instead of \, within one \
https://docs.gradle.org/6.8/userguide/java_testing.html#mergereruns
Contributor guide
Assessment
This issue has not been assessed yet.