apache / apache/maven-surefire

[SUREFIRE-1930] JUnit47ParallelIT.forcedShutdownVerifyingLogs is flaky

Open
#2,902 0 comments 0 reactions 0 assignees View on GitHub
bug priority:minor
Dominant language
Java
Stars
461
Forks
588
Avg merge
1d 8h
Merged PRs (30d)
19

Description

**[Alexander Kriegisch](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=kriegaex)** opened **[SUREFIRE-1930](https://issues.apache.org/jira/browse/SUREFIRE-1930?redirect=false)** and commented

While working on an unrelated PR, in one case `JUnit47ParallelIT.forcedShutdownVerifyingLogs` failed, see [this GitHub build log](https://github.com/apache/maven-surefire/runs/2982270818?check_suite_focus=true#step:4:4010):

```
Error: Tests run: 59, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 344.612 s <<< FAILURE! - in org.apache.maven.surefire.its.JUnit47ParallelIT
Error: org.apache.maven.surefire.its.JUnit47ParallelIT.forcedShutdownVerifyingLogs Time elapsed: 4.94 s <<< ERROR!
org.apache.maven.surefire.its.fixture.SurefireVerifierException: org.apache.maven.it.VerificationException: Text not found in log: These tests are incomplete:
at org.apache.maven.surefire.its.JUnit47ParallelIT.forcedShutdownVerifyingLogs(JUnit47ParallelIT.java:523)
Caused by: org.apache.maven.it.VerificationException: Text not found in log: These tests are incomplete:
at org.apache.maven.surefire.its.JUnit47ParallelIT.forcedShutdownVerifyingLogs(JUnit47ParallelIT.java:523)
```

The other builds passed. I am not sure if the test is flaky on a more regular basis, but I have seen it fail more than once, e.g. also [in this build](https://github.com/apache/maven-surefire/runs/2979364889?check_suite_focus=true#step:4:4010). Both times, it ocurred on MacOS, but that could be by chance.

[tibordigana](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=tibordigana) requested to handle this issue separately from the PR, and I am happy to comply.

---

**Affects:** 3.0.0-M5

Contributor guide

Open the contributing guide

Research direction

Start with JUnit47ParallelIT.forcedShutdownVerifyingLogs in JUnit47ParallelIT.java at line 523, then compare the referenced MacOS build logs where the expected text is missing. Reproduce the test failure and identify why "These tests are incomplete:" is not present in the log. Done means the integration test no longer flakes and still verifies the expected shutdown log output.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.