[SUREFIRE-1433] rerunFailingTestsCount does not help with crashes

**[Jesse Glick](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=jglick@netbeans.org)** opened **[SUREFIRE-1433](https://issues.apache.org/jira/browse/SUREFIRE-1433?redirect=false)** and commented

I have a CI job running lots of very slow tests, and am using `-Dsurefire.rerunFailingTestsCount=2 -Dmaven.test.failure.ignore -Dsurefire.timeout=7200 -DreuseForks=false -DforkCount=2` to run Maven. It will typically run for about three hours, collecting a lot of passing tests, a few flaking tests with subsequent passes, and a few true failures. But then it will abruptly halt with the infamous

```none
Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project ...: ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
```

The timing is unpredictable but the build seems to usually fail this way; sometimes I get lucky and it runs through all the tests, taking a number of hours. Diagnosing the crash is not likely to be straightforward, and frankly I do not care much—I just want Surefire to try again. Yet the `rerunFailingTestsCount` flag does not accomplish this—it only retries normal failures.

You can see this from running the attached project. While you might expect it to go through 100 runs of the test, some failing politely and some crashing, instead you see something like

```none
...
--- maven-surefire-plugin:2.20.1:test (default-test) @ demo ---

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running demo.CrashingTest
failing
failing
failing
crashing

Results:

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

------------------------------------------------------------------------
BUILD FAILURE
------------------------------------------------------------------------
Total time: 2.456 s
Finished at: 2017-10-07T15:11:04-04:00
Final Memory: 11M/222M
------------------------------------------------------------------------
Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on project demo: There are test failures.
...
```

---

**Affects:** 2.20.1

**Attachments:**
- [demo.zip](https://issues.apache.org/jira/secure/attachment/12890876/demo.zip) (_1.44 kB_)

Start by reproducing the crash-retry behavior with the attached demo.zip and the Maven Surefire options shown in the issue, especially rerunFailingTestsCount and reuseForks=false. Trace how a forked VM termination is reported versus a normal test failure. Done means crashed test forks are retried as requested without losing the existing failure reporting.

Written by the indexing model from the issue text.