apache / apache/maven-surefire
[SUREFIRE-2102] Different Behaviour while using junit-jupiter-engine/jupiter-vintage-engine/testng-engine
- Dominant language
- Java
- Stars
- 461
- Forks
- 588
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 19
Description
**[Karl Heinz Marbaise](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=khmarbaise)** opened **[SUREFIRE-2102](https://issues.apache.org/jira/browse/SUREFIRE-2102?redirect=false)** and commented
I have a full working example which executes three different kind of tests:
* Junit Jupiter based
* JUnit 4 based
* TestNG based
what I can observe there seemed to be a difference in behaviour.
```
[INFO] Running com.soebes.youtube.maven.episodes.ep3.jupiter.FractionJUnitJupiterTest
FractionJUnitJupiterTest.beforeAll
FractionJUnitJupiterTest.beforeEach
FractionJUnitJupiterTest.first_add
FractionJUnitJupiterTest.afterEach
FractionJUnitJupiterTest.afterAll
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 s - in com.soebes.youtube.maven.episodes.ep3.jupiter.FractionJUnitJupiterTest
[INFO] Running com.soebes.youtube.maven.episodes.ep3.junit.FractionJUnit4Test
FractionJUnit4Test.beforeClass
FractionJUnit4Test.before
FractionJUnit4Test.first_add
FractionJUnit4Test.after
FractionJUnit4Test.afterClass
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 s - in com.soebes.youtube.maven.episodes.ep3.junit.FractionJUnit4Test
[INFO] Running com.soebes.youtube.maven.episodes.ep3.testng.FractionTestNGTest
FractionTestNGTest.beforeClass
FractionTestNGTest.beforeMethod
FractionTestNGTest.first_add
FractionTestNGTest.afterMethod
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 s - in com.soebes.youtube.maven.episodes.ep3.testng.FractionTestNGTest
FractionTestNGTest.afterClass
[DEBUG] Closing the fork 1 after saying GoodBye.
```
You can see that the output of the `afterClass` is after the output of the test result. But only for TestNG while for JUnit 4 and JUnit Jupiter it looks the same.
The same happens for using failsafe in the same example project.
I made a full working reproducible example: https://github.com/khmarbaise/sffs-bugs (Branch: main). This requires JDK 17 (but should also work with JDK11)...
The repository contains two full DEBUG logoutput files: `mvn.log` (running `mvn verify`) and `mvn-test.log` only for running unit test via `mvn test`
---
**Affects:** 3.0.0-M7
Contributor guide
Research direction
Start with the reproducible example in the sffs-bugs repository and run it with JDK 17 using mvn test and mvn verify. Compare mvn.log and mvn-test.log, focusing on why TestNG afterClass output appears after the test result while the JUnit Jupiter and JUnit 4 output does not; done means the lifecycle output is consistent across engines.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100