apache / apache/maven-surefire

[SUREFIRE-1985] While running test cases if a test case fails, need to kill the fork and run subsequent testcases(including the failed one) with a new fork

Open
#2,861 5 comments 0 reactions 0 assignees View on GitHub
priority:major
Dominant language
Java
Stars
461
Forks
588
Avg merge
1d 8h
Merged PRs (30d)
19

Description

**[Sanidhya Vijaivargia](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=JIRAUSER284086)** opened **[SUREFIRE-1985](https://issues.apache.org/jira/browse/SUREFIRE-1985?redirect=false)** and commented

After running a particular module/project if the test fails even after rerunning it N specified times the subsequent modules fail automatically without running them.

Following is a use-case for this

```java
import org.junit.Assert;
import org.junit.Test;

public class surefireTest {
private static boolean flag = true;

@Test
public void test1() {
Assert.assertTrue(flag);
flag = false;
}

@Test
public void test2() {
test1();
}

@Test
public void test3() {
test1();
}
}

```

Running the above code using surefire using \1\ & \3\ should ideally be able to pass all the testcases, but in our case it is able to pass just testcase 1. Is it possible to have a completely isolated environment (free from pollution from other testcases) for all 3 testcases, that is as soon as a testcase fails, we kill the current fork and start a new fork and retry for the failed testcase and so on.

Please find the observed result attached below.
Thanks!

---

**Attachments:**
- [Screenshot 2022-01-25 at 10.36.34 AM.png](https://issues.apache.org/jira/secure/attachment/13039328/Screenshot+2022-01-25+at+10.36.34+AM.png) (_28.42 kB_)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Java example with forkCount set to 1 and rerunFailingTestsCount set to 3 in Maven Surefire. Trace how failed tests and subsequent test cases are handled across fork reuse, then verify that a failed test and later tests run in newly isolated forks and can pass without inherited state.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.