[SUREFIRE-1866] Nested tests are reported ambiguously

**[Marc Philipp](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=marcphilipp)** opened **[SUREFIRE-1866](https://issues.apache.org/jira/browse/SUREFIRE-1866?redirect=false)** and commented

In 3.0.0-M4 the way nested tests and test suites are reported changed. I've prepared a small reproducer project at https://github.com/marcphilipp/gradle-sandbox/tree/master/maven-surefire-nested-test-reproducer

## Expected behavior (with 3.0.0-M3)

Run `mvn clean test -Dmaven.surefire.version=3.0.0-M3`.

### Console output

```
[INFO] Running org.example.FailingTests
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.024 s <<< FAILURE! - in org.example.FailingTests
[ERROR] test Time elapsed: 0.021 s <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: but was:
at org.example.AbstractTests$Inner.test(AbstractTests.java:14)

[INFO] Running org.example.SuccessfulTests
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.example.SuccessfulTests
```

### XML Reports

#### TEST-org.example.FailingTests.xml

```xml

[...]


but was:
at org.example.AbstractTests$Inner.test(AbstractTests.java:14)
]]>

```

#### TEST-org.example.SuccessfulTests.xml

```xml

[...]

```

## Actual behavior (with 3.0.0-M4 and 3.0.0-M5)

Run `mvn clean test`.

### Console output

```
[INFO] Running org.example.FailingTests
[INFO] Running org.example.AbstractTests$Inner
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.037 s <<< FAILURE! - in org.example.AbstractTests$Inner
[ERROR] org.example.AbstractTests$Inner.test Time elapsed: 0.02 s <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: but was:
at org.example.AbstractTests$Inner.test(AbstractTests.java:14)

[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.046 s - in org.example.FailingTests
[INFO] Running org.example.SuccessfulTests
[INFO] Running org.example.AbstractTests$Inner
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 s - in org.example.AbstractTests$Inner
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 s - in org.example.SuccessfulTests
```

### XML Reports

#### TEST-org.example.FailingTests.xml

```xml

[...]

```

#### TEST-org.example.SuccessfulTests.xml

```xml

[...]

```

#### TEST-org.example.AbstractTests$Inner.xml

```xml

[...]


but was:
at org.example.AbstractTests$Inner.test(AbstractTests.java:14)
]]>

```

## Summary

The main issue is that the reporting is now ambiguous, i.e. there's no way to tell which test failed.

---

**Affects:** 3.0.0-M4, 3.0.0-M5

2 votes, 4 watchers

Use the linked gradle-sandbox maven-surefire-nested-test-reproducer and run `mvn clean test` with Surefire 3.0.0-M3, then with the default version, comparing console output and the TEST-org.example.*.xml reports. Done means nested test failures are attributable to the correct test and the suite reports are no longer ambiguous.

Written by the indexing model from the issue text.