apache / apache/maven-surefire

[SUREFIRE-2213] @Categories (JUnit4) are handeld differnt than JUnit's built-in Categories-Runner

Open
#2,918 0 comments 0 reactions 0 assignees View on GitHub
bug priority:major
Dominant language
Java
Stars
461
Forks
588
Avg merge
1d 8h
Merged PRs (30d)
19

Description

**[Christian Nüssgens](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=JIRAUSER288303)** opened **[SUREFIRE-2213](https://issues.apache.org/jira/browse/SUREFIRE-2213?redirect=false)** and commented

With SUREFIRE-1695 a change was introduced which modified the behavior how `` and `` are handled.

Assume the following Test Classes

```java
@Category(CategoryA.class)
public abstract class AbstractTest {
}

public class TestA extends AbstractTest {
@Test
public void testIt() throws Exception {
//do test
}
}

@Category(CategoryPlayground.class)
public class PlaygroundTestA extends TestA {

@Test
public void testIt() throws Exception {
// do nasty stuff like while(true)
}
}
```

The built-in categories Runner behaves as follows

```java
@RunWith(Categories.class)
@IncludeCategory({CategoryA.class})
@SuiteClasses({TestA.class, PlaygroundTestA.class})
public class TheSuite {
}
```

If run: only `TestA` is executed. The "closest" Categories declaration wins, although the Categories-annotation itself is "inheritable".

Surefire does behave differently:

```xml

CategoryA

```

If run (`mvn test`) surefire will execute `TestA` and `PlaygroundTestA` (since surefire 3.0.0-M5)

We were quite surprised that this behavior changed from surefire 2.x to 3.x and is now not reflecting the way it is implemented in JUnit itself.

I'm currently not sure _who_ is already relying on the _new_ behavior.
Is it possible to restore the "old" behavior? Or would it be a convenient approach to make it?configurable.

---

**Affects:** 3.0.0-M5, 3.0.0-M6, 3.0.0-M7, 3.0.0-M8, 3.0.0-M9, 3.0.0, 3.1.0, 3.1.2, 3.2.1, 3.2.2

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the behavior change introduced by SUREFIRE-1695, then reproduce the example with the shown JUnit4 classes and CategoryA configuration using mvn test. Determine whether the expected outcome is to restore JUnit's closest-category behavior or make the behavior configurable, and add coverage for the decided result.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.