[SUREFIRE-779] Surefire reports wrong number of failed tests when using JUnit's ErrorCollector rule

**[Lieven Doclo](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=doclolieven)** opened **[SUREFIRE-779](https://issues.apache.org/jira/browse/SUREFIRE-779?redirect=false)** and commented

When running a test that contains an ErrorCollector, test failures are not correct when an errorcollector contains failures. For example:

```
public class ExampleTest {
@Rule
public ErrorCollector errorCollector = new ErrorCollector();

@Test
public void testWithErrorCollector() {
errorCollector.checkThat(true, is(false));
errorCollector.checkThat(3, is(4));
}
}
```

Surefire will reports 2 tests run (of which all failed). However, when you fix the same test so that no checks fail, Surefire will report 1 test run.

This is incorrect, as this will result in false information in systems like Hudson or Sonar, the number of testcase should not fluctuate. Surefire should only report 1 test run, as that reflects the actual situation.

---

**Affects:** 2.10

**Issue Links:**
- [SUREFIRE-841](https://issues.apache.org/jira/browse/SUREFIRE-841) Incorrect Test Run Count
(_**"Blocked"**_)
- [SUREFIRE-1172](https://issues.apache.org/jira/browse/SUREFIRE-1172) Incompatibility with MultipleFailureException
(_**"is duplicated by"**_)
- [SUREFIRE-841](https://issues.apache.org/jira/browse/SUREFIRE-841) Incorrect Test Run Count

Start by reproducing the ExampleTest with JUnit's ErrorCollector and compare Surefire's reported test count when its checks fail versus pass. Review the related SUREFIRE-841 and SUREFIRE-1172 issues before changing behavior; done means one test is reported in both cases, matching the actual test count.

Written by the indexing model from the issue text.