apache / apache/maven-surefire
3.6.0 regression: Spock specs abort discovery (Tests run: 0) when `<excludedGroups>` is set and JUnit 4 is on the test classpath
- Dominant language
- Java
- Stars
- 461
- Forks
- 588
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 19
Description
### Affected version
3.6.0 and 3.6.0-M1. Works with 3.5.6.
### Bug description
> *Drafted with AI assistance. I built and ran the reproducer and verified the analysis myself.*
A Spock specification aborts JUnit Platform discovery when `` is configured and the JUnit 4 jar is on the test classpath. The fork dies before any test runs, so the module reports `Tests run: 0` — plain Jupiter tests in the same module do not run either.
Minimal reproducer: https://github.com/MenschNestor/surefire-spock-repro (one Spock feature, one Jupiter test)
```
mvn test -Dsurefire.version=3.5.6 # Tests run: 2, BUILD SUCCESS
mvn test -Dsurefire.version=3.6.0 # Tests run: 0, BUILD FAILURE
```
```
[INFO] --- surefire:3.6.0:test (default-test) @ surefire-spock-repro ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[ERROR] Could not find method with name [addition works] in class [com.example.ExampleSpec].
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[ERROR] There was an error in the forked process
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process
```
All three conditions are jointly necessary:
| surefire | `excludedGroups` | `junit:junit` on classpath | tests run |
| --- | --- | --- | ---: |
| 3.5.6 | `slow` | yes | 2 |
| 3.6.0 | *(empty)* | yes | 2 |
| 3.6.0 | `slow` | no | 2 |
| 3.6.0 | `slow` | yes | **0, BUILD FAILURE** |
| 3.6.0-M1 | `slow` | yes | **0, BUILD FAILURE** |
## Cause
In `JUnitPlatformProvider#newFilters`, the exclude-category filter is added whenever `org.junit.experimental.categories.Category` is loadable:
```java
if (!useTestNG) {
Optional> categoryClass = getCategoryClass();
if (categoryClass.isPresent()) {
getPropertiesList(EXCLUDEDGROUPS_PROP)
.map(strings -> getExcludeCategoryFilter(strings, categoryClass))
.ifPresent(filters::add);
}
}
```
Unlike the include branch just above it, this is not gated on `JUNIT_VINTAGE_DETECTED`, so having the JUnit 4 jar anywhere on the test classpath is enough — the Vintage engine does not have to be present, and in the reproducer it is not. The resulting `PostDiscoveryFilter` is applied to descriptors from every engine, and to read the `@Category` annotation it calls
```java
methodSource.get().getJavaMethod()
```
in `getExcludeCategoryFilter`, and identically in `getIncludeCategoryFilter`.
Spock's `MethodSource` carries the *feature name*, not a JVM method name: Spock renames feature methods to `$spock_feature_N_M` and keeps the display name in `@FeatureMetadata`. In the reproducer the feature is `def "addition works"()`, so the reported name contains a space and can never resolve reflectively. `MethodSource#getJavaMethod` throws `PreconditionViolationException`, which escapes discovery and takes the fork down.
3.5.6 mapped `groups`/`excludedGroups` to `TagFilter.includeTags`/`excludeTags` only, with no reflection, so a `MethodSource` that does not correspond to a real method was harmless.
This is not specific to Spock. Any JUnit Platform engine whose `MethodSource` does not map one-to-one onto a reflective `Method` will hit it.
## Possible fixes
Gating both category filters on `JUNIT_VINTAGE_DETECTED` would confine them to descriptors that can actually carry `@Category`. Independently, resolving the method defensively — treating an unresolvable `MethodSource` as "no category" rather than letting the exception escape — would stop a discovery-time reflection failure from killing the entire fork.
## Impact
In our multi-module build this turns every module containing a Spock specification into `Tests run: 0` plus a hard failure — five modules.
Contributor guide
Research direction
Start at JUnitPlatformProvider#newFilters and the getIncludeCategoryFilter/getExcludeCategoryFilter paths described in the issue. Run the linked reproducer with Surefire 3.5.6 and 3.6.0, then verify that excludedGroups with JUnit 4 present no longer aborts discovery and that both Spock and Jupiter tests run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 65/100