apache / apache/maven-surefire

[SUREFIRE-1812] Skipped tests (by annotation @EnabledIfEnvironmentVariable on the test class) are not reported

Open
#3,039 9 comments 0 reactions 0 assignees View on GitHub
bug priority:major
Dominant language
Java
Stars
461
Forks
588
Avg merge
1d 8h
Merged PRs (30d)
19

Description

**[Jiri Ondrusek](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=jondruse)** opened **[SUREFIRE-1812](https://issues.apache.org/jira/browse/SUREFIRE-1812?redirect=false)** and commented

I'm upgrading surefire plugin from 2.22.2 and there is a different behavior started from 3.0.0-M4.

When the test class is ignored by using `@EnabledIfEnvironmentVariable`(...), final report shows bad information: (3.0.0.-M4, M5, master)

> [INFO] Results:
> [INFO]
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

The same test executed with 3.0.0-M3 shows:

> [INFO] Results:
> [INFO]
> [WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1

---

**Affects:** 3.0.0-M4, 3.0.0-M5

2 votes, 4 watchers

Contributor guide

Open the contributing guide

Research direction

Start in the Maven Surefire test-reporting path by reproducing the case where a test class uses @EnabledIfEnvironmentVariable and comparing Surefire 3.0.0-M3 with M4, M5, or master. Done means the final Results report records the class-level skipped test as one skipped test rather than reporting zero tests run.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.