apache / apache/maven-surefire

[SUREFIRE-1953] Junit 5 in parallel execution mode confuses Surefire Console logs

Open
#2,901 1 comment 0 reactions 0 assignees View on GitHub
bug priority:major
Dominant language
Java
Stars
461
Forks
588
Avg merge
1d 8h
Merged PRs (30d)
19

Description

**[Praveen Narala](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=praveenreddynarala)** opened **[SUREFIRE-1953](https://issues.apache.org/jira/browse/SUREFIRE-1953?redirect=false)** and commented

When I use Junit 5 to run the tests in parallel (not using Surefire Forking), the console logs are confusing when the test failed.

As per Junit 5 documentation, when you use Junit 5 for parallelism, the tests will run using different Forking threads in the same process. And, when there is a test failure, Surefire is logging respective test failure logs. Instead, it is just picking random logs from different threads.

**Would it be possible to get the right logs for failed tests?**

 

**Junit 5 Configuration from Surefire Plugin:**

 

 

```java
Tests run:
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @ api-test ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.test.abc.ABCTests
[INFO] Running com.test.abc.XYZTests
[INFO] Running com.test.abc.ATests
[INFO] Running com.test.abc.GetLogsTests
[INFO] Running com.test.abc.FoldersTests
[INFO] Running com.test.abc.NameTests
```

 

 

**Example Logs from Console:**

 

```java
NFO |1026-103227268|ForkJoinPool-1-worker-3|abc.ABCTests||||||About to execute test [delegatedUserAccount] - GUID [random-guid] and data [[1] com.test.abc.ABCTestData@37589144]
INFO |1026-103227268|ForkJoinPool-1-worker-13|abc.XYZTests||||||About to execute test [getTestId] - GUID [random-guid] and data [[1] com.test.abc.data.XYZTestData@4c506080]
INFO |1026-103227268|ForkJoinPool-1-worker-5|abc.ATests||||||About to execute test [getFolderTest] - GUID [0e1141b0-3445-36f7-879a-2f12f3de87a7] and data [[1] com.test.abc.data.FoldersTestData@484d6599]
INFO |1026-103227268|ForkJoinPool-1-worker-7|abc.GetLogsTests||||||About to execute test [getIds] - GUID [random-guid] and data [[1] com.test.abc.data.GetLogsTestData@4cf26b0c]
```

 

_When there is any test failure from the execution, we see logs from all different threads like below..._

 

 

```java
INFO |1026-103227268|ForkJoinPool-1-worker-3|abc.ABCTests||||||About to execute test [delegatedUserAccount] - GUID [random-guid] and data [[1] com.test.abc.ABCTestData@37589144]
INFO |1026-103227268|ForkJoinPool-1-worker-13|abc.XYZTests||||||About to execute test [getTestId] - GUID [random-guid] and data [[1] com.test.abc.data.XYZTestData@4c506080]
INFO |1026-103227268|ForkJoinPool-1-worker-5|abc.ATests||||||About to execute test [getFolderTest] - GUID [0e1141b0-3445-36f7-879a-2f12f3de87a7] and data [[1] com.test.abc.data.FoldersTestData@484d6599]
INFO |1026-103227268|ForkJoinPool-1-worker-7|abc.GetLogsTests||||||About to execute test [getIds] - GUID [random-guid] and data [[1] com.test.abc.data.GetLogsTestData@4cf26b0c]
```

 

**Expected Behavior**

It would be good to have console logs at the Test case level rather than picking up from random threads though we run tests using Junit 5 or Surefire Forks (JVMs).

 

**Steps To Reproduce**

Use Junit 5 Configuration from Maven Surefire Plugin v3.0.0-M5 with JDK11.

```java

maven-surefire-plugin
${surefire.plugin.version}

false


**/*Test.java
**/*Tests.java


${Tags}
Skip, WIP


junit.jupiter.conditions.deactivate = *
junit.jupiter.extensions.autodetection.enabled = true
junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = concurrent
junit.jupiter.execution.parallel.config.strategy=fixed
junit.jupiter.execution.parallel.config.fixed.parallelism=6
junit.jupiter.testinstance.lifecycle.default = per_class



COMPAT

```

Run the tests using the maven command to run the tests in parallel using Tags...

```java
mvn -Dgroups=TestingTag
```

---
No further details from [SUREFIRE-1953](https://issues.apache.org/jira/browse/SUREFIRE-1953?redirect=false)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue with the Maven Surefire Plugin v3.0.0-M5 configuration, JDK 11, JUnit 5 parallel execution, and the provided Maven command. Trace how failed-test console output is associated with parallel threads; done means a failure reports the logs for the corresponding test rather than unrelated thread output, with regression coverage added where the behavior is implemented.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.