apache / apache/maven-surefire
[SUREFIRE-1953] Junit 5 in parallel execution mode confuses Surefire Console logs
- Dominant language
- Java
- Stars
- 461
- Forks
- 588
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 19
Description
**[Praveen Narala](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=praveenreddynarala)** opened **[SUREFIRE-1953](https://issues.apache.org/jira/browse/SUREFIRE-1953?redirect=false)** and commented
When I use Junit 5 to run the tests in parallel (not using Surefire Forking), the console logs are confusing when the test failed.
As per Junit 5 documentation, when you use Junit 5 for parallelism, the tests will run using different Forking threads in the same process. And, when there is a test failure, Surefire is logging respective test failure logs. Instead, it is just picking random logs from different threads.
**Would it be possible to get the right logs for failed tests?**
**Junit 5 Configuration from Surefire Plugin:**
```java
Tests run:
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @ api-test ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.test.abc.ABCTests
[INFO] Running com.test.abc.XYZTests
[INFO] Running com.test.abc.ATests
[INFO] Running com.test.abc.GetLogsTests
[INFO] Running com.test.abc.FoldersTests
[INFO] Running com.test.abc.NameTests
```
**Example Logs from Console:**
```java
NFO |1026-103227268|ForkJoinPool-1-worker-3|abc.ABCTests||||||About to execute test [delegatedUserAccount] - GUID [random-guid] and data [[1] com.test.abc.ABCTestData@37589144]
INFO |1026-103227268|ForkJoinPool-1-worker-13|abc.XYZTests||||||About to execute test [getTestId] - GUID [random-guid] and data [[1] com.test.abc.data.XYZTestData@4c506080]
INFO |1026-103227268|ForkJoinPool-1-worker-5|abc.ATests||||||About to execute test [getFolderTest] - GUID [0e1141b0-3445-36f7-879a-2f12f3de87a7] and data [[1] com.test.abc.data.FoldersTestData@484d6599]
INFO |1026-103227268|ForkJoinPool-1-worker-7|abc.GetLogsTests||||||About to execute test [getIds] - GUID [random-guid] and data [[1] com.test.abc.data.GetLogsTestData@4cf26b0c]
```
_When there is any test failure from the execution, we see logs from all different threads like below..._
```java
INFO |1026-103227268|ForkJoinPool-1-worker-3|abc.ABCTests||||||About to execute test [delegatedUserAccount] - GUID [random-guid] and data [[1] com.test.abc.ABCTestData@37589144]
INFO |1026-103227268|ForkJoinPool-1-worker-13|abc.XYZTests||||||About to execute test [getTestId] - GUID [random-guid] and data [[1] com.test.abc.data.XYZTestData@4c506080]
INFO |1026-103227268|ForkJoinPool-1-worker-5|abc.ATests||||||About to execute test [getFolderTest] - GUID [0e1141b0-3445-36f7-879a-2f12f3de87a7] and data [[1] com.test.abc.data.FoldersTestData@484d6599]
INFO |1026-103227268|ForkJoinPool-1-worker-7|abc.GetLogsTests||||||About to execute test [getIds] - GUID [random-guid] and data [[1] com.test.abc.data.GetLogsTestData@4cf26b0c]
```
**Expected Behavior**
It would be good to have console logs at the Test case level rather than picking up from random threads though we run tests using Junit 5 or Surefire Forks (JVMs).
**Steps To Reproduce**
Use Junit 5 Configuration from Maven Surefire Plugin v3.0.0-M5 with JDK11.
```java
maven-surefire-plugin
${surefire.plugin.version}
false
**/*Test.java
**/*Tests.java
${Tags}
Skip, WIP
junit.jupiter.conditions.deactivate = *
junit.jupiter.extensions.autodetection.enabled = true
junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = concurrent
junit.jupiter.execution.parallel.config.strategy=fixed
junit.jupiter.execution.parallel.config.fixed.parallelism=6
junit.jupiter.testinstance.lifecycle.default = per_class
COMPAT
```
Run the tests using the maven command to run the tests in parallel using Tags...
```java
mvn -Dgroups=TestingTag
```
---
No further details from [SUREFIRE-1953](https://issues.apache.org/jira/browse/SUREFIRE-1953?redirect=false)
Contributor guide
Research direction
Start by reproducing the issue with the Maven Surefire Plugin v3.0.0-M5 configuration, JDK 11, JUnit 5 parallel execution, and the provided Maven command. Trace how failed-test console output is associated with parallel threads; done means a failure reports the logs for the corresponding test rather than unrelated thread output, with regression coverage added where the behavior is implemented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100