jenkinsci / jenkinsci/junit-plugin

Support <flakyFailure> and <flakyError> in junit xml report

Open
#237 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
Java
Stars
87
Forks
351
Avg merge
1d 19h
Merged PRs (30d)
2

Description

Both Maven and Gradle support automatically re-running tests on failure and marking them as flaky if they pass on the retry. For better or worse, flaky tests are a common problem for integration and system tests. Marking such tests as flaky would be very helpful.

The [Flaky Test Handler](https://plugins.jenkins.io/flaky-test-handler/) plugin has this functionality and more, but it was last released 5 years ago and not actively maintained. It would be great if the basic reporting functionality was part of the core junit plugin.

Relevant documentation for Maven surefire:

> flakyFailure and flakyError elements will be used in the generated xml report to include information of each failing re-runs.
https://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html

Similarly, Gradle:

> When mergeReruns is enabled, if a test fails but is then retried and succeeds, its failures will be recorded as \ instead of \, within one \
https://docs.gradle.org/6.8/userguide/java_testing.html#mergereruns

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.