[SUREFIRE-1970] rerunFailingTestsCount option marks some spock tests as false positive

**[Casey Jones](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=JIRAUSER282498)** opened **[SUREFIRE-1970](https://issues.apache.org/jira/browse/SUREFIRE-1970?redirect=false)** and commented

I have a project which uses [Spock Framework](https://spockframework.org/) for tests. There is a single test which intentionally fails, no matter how many times it was ran:

```java
class FalsePositiveTest extends Specification {
def "test"() {
expect:
1 == number
where:
number << [2, 3]
}
}
```

When test is executed with `mvn test` command, the build fails, as expected. 

However, when using surefire's [rerunFailingTestsCount](https://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html) option, the test in question is executed two times, as expected, but then it is marked as flaky, and the build is successful:

```java
mvn test -Dsurefire.rerunFailingTestsCount=1
...
Warning: Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Flakes: 1
126[INFO]
127[INFO] ------------------------------------------------------------------------
128[INFO] BUILD SUCCESS
129[INFO] ------------------------------------------------------------------------
```

Please check the links below for a full build log.

This issue happens only when using Spock's data-driven tests with a [where block](https://spockframework.org/spock/docs/1.0/data_driven_testing.html) and surefire's `rerunFailingTestsCount` option together. It seems like there is something wrong in the way surefire detects if test has failed or passed.

I asked this [question on StackOverflow](https://stackoverflow.com/questions/70454980/spock-data-driven-tests-surefire-rerunfailingtestscount-gives-false-positive-t) and Spock's maintainer suggested to raise an issue in surefire's bugtracker.

Here is a simple github project which reproduces the issue: https://github.com/streetturtle/FalsePositiveTest

Please check build logs of a github action here: https://github.com/streetturtle/FalsePositiveTest/runs/4613803891?check_suite_focus=true#step:4:128

---

**Affects:** 3.0.0-M4, 3.0.0-M5

1 votes, 2 watchers

Start with the linked FalsePositiveTest reproducer and run `mvn test` with `-Dsurefire.rerunFailingTestsCount=1`, comparing its output with the provided build log. Trace how Surefire reports Spock data-driven tests with a `where` block, and consider the issue complete when the intentionally failing test still fails the build after reruns.

Written by the indexing model from the issue text.