apache / apache/maven-surefire
[SUREFIRE-1930] JUnit47ParallelIT.forcedShutdownVerifyingLogs is flaky
- Dominant language
- Java
- Stars
- 461
- Forks
- 588
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 19
Description
**[Alexander Kriegisch](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=kriegaex)** opened **[SUREFIRE-1930](https://issues.apache.org/jira/browse/SUREFIRE-1930?redirect=false)** and commented
While working on an unrelated PR, in one case `JUnit47ParallelIT.forcedShutdownVerifyingLogs` failed, see [this GitHub build log](https://github.com/apache/maven-surefire/runs/2982270818?check_suite_focus=true#step:4:4010):
```
Error: Tests run: 59, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 344.612 s <<< FAILURE! - in org.apache.maven.surefire.its.JUnit47ParallelIT
Error: org.apache.maven.surefire.its.JUnit47ParallelIT.forcedShutdownVerifyingLogs Time elapsed: 4.94 s <<< ERROR!
org.apache.maven.surefire.its.fixture.SurefireVerifierException: org.apache.maven.it.VerificationException: Text not found in log: These tests are incomplete:
at org.apache.maven.surefire.its.JUnit47ParallelIT.forcedShutdownVerifyingLogs(JUnit47ParallelIT.java:523)
Caused by: org.apache.maven.it.VerificationException: Text not found in log: These tests are incomplete:
at org.apache.maven.surefire.its.JUnit47ParallelIT.forcedShutdownVerifyingLogs(JUnit47ParallelIT.java:523)
```
The other builds passed. I am not sure if the test is flaky on a more regular basis, but I have seen it fail more than once, e.g. also [in this build](https://github.com/apache/maven-surefire/runs/2979364889?check_suite_focus=true#step:4:4010). Both times, it ocurred on MacOS, but that could be by chance.
[tibordigana](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=tibordigana) requested to handle this issue separately from the PR, and I am happy to comply.
---
**Affects:** 3.0.0-M5
Contributor guide
Research direction
Start with JUnit47ParallelIT.forcedShutdownVerifyingLogs in JUnit47ParallelIT.java at line 523, then compare the referenced MacOS build logs where the expected text is missing. Reproduce the test failure and identify why "These tests are incomplete:" is not present in the log. Done means the integration test no longer flakes and still verifies the expected shutdown log output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100