apache / apache/maven-surefire

[SUREFIRE-2194] JUnit 5: Implementing tests in test class and nested class results in incorrect reporting

Open
#2,682 6 comments 0 reactions 0 assignees View on GitHub
bug priority:minor
Dominant language
Java
Stars
461
Forks
588
Avg merge
1d 8h
Merged PRs (30d)
19

Description

**[Jochen Kraushaar](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=JIRAUSER302166)** opened **[SUREFIRE-2194](https://issues.apache.org/jira/browse/SUREFIRE-2194?redirect=false)** and commented

### Example Test Class

```java
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.assertTrue;

class ExampleTest {

@Test
void outerTest() {
assertTrue(true);
}

@Nested
class InnerExampleTest {
@Test
void innerTest() {
assertTrue(true);
}
}
}
```

### Expected Result

**One** `TEST-ExampleTest.xml` file containing the results of both tests or **two** files (`{}TEST-ExampleTest.xml{`} and `{}TEST-ExampleTest$InnerExampleTest.xml{`}) containing the results of their tests.

### Actual Result

* `TEST-ExampleTest.xml` does not contain any test case.
* `TEST-ExampleTest$InnerExampleTest.xml` contains results for both test cases.

### Workaround

Do not put any tests into the parent class if nested classes are used.

### Analysis

`org.apache.maven.surefire.junitplatform.RunListenerAdapter` does not distinguish between the parent and the nested classes. Both are reported as new test sets.

`org.apache.maven.plugin.surefire.report.TestSetRunListener` does not support nested test sets, so the test set of the nested class overrides the test set of the parent class.

---

**Affects:** 3.1.2

Contributor guide

Open the contributing guide

Research direction

Start with the example JUnit 5 test in the issue and inspect org.apache.maven.surefire.junitplatform.RunListenerAdapter and org.apache.maven.plugin.surefire.report.TestSetRunListener. Reproduce the generated reports, then trace how the parent and nested test sets are recorded. Done means the parent and nested tests appear in the expected one or two XML report files without either test set losing results.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.