apache / apache/maven-surefire

[SUREFIRE-1925] Bogus skipped test reported for class tested immediately after a JUnit 5 @Disabled class

Open
#2,927 0 comments 0 reactions 0 assignees View on GitHub
bug priority:major
Dominant language
Java
Stars
461
Forks
588
Avg merge
1d 8h
Merged PRs (30d)
19

Description

**[Benjamin Morgan](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=labalius)** opened **[SUREFIRE-1925](https://issues.apache.org/jira/browse/SUREFIRE-1925?redirect=false)** and commented

If a test class is annotated with org.junit.jupiter.api.Disabled (and is therefore not tested) then the next test class that Maven processes will be reported as having a skipped test that does not in fact exist. This manifests in the .txt file for the second test class in the surefire-reports folder. For example, if the second test class has precisely two methods annotated with org.junit.jupiter.api.Test, neither of which is annotated with org.junit.jupiter.api.Disabled, and if both tests pass then the test report will be like so:

---

Test set: com.example.SecondTest

---

Tests run: 3, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.025 s - in com.example.SecondTest

It does not matter whether or not the second test class is in the same package as the first. If the two test methods in the second test class are annotated with Disabled, then the Skipped count is reported as 3.

---

**Affects:** 3.0.0-M4, 3.0.0-M5

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with two JUnit 5 classes, making the first class @Disabled and the second contain the described @Test methods. Compare the second class's .txt file in surefire-reports with the actual test results; done means the skipped count reflects only tests actually skipped in that class.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.