Reaching mocha timeout leads to incorrect coverage (vscode extension testing)
- Dominant language
- JavaScript
- Stars
- 2.1k
- Forks
- 99
- PR merge metrics
- No merged PRs in 30d
Description
* **Version**: v18.7.0
* **Platform**: Linux x86_64 (should affect all)
This is more of a note rather than an bug report and would probably be more appropriate as a GitHub Discussion if they were enabled.
We just starting using `c8` for coverage in our VS Code extension https://github.com/fortran-lang/vscode-fortran-support and what I noticed was that if we hit our `timeout` in our `mocha` unittests `c8` carries on with the coverage report as if the unittests have run succesully. Maybe it would be beneficial if `c8` threw some kind of warning that the coverage **might** be inaccurate because you reached `timeout`.
In our case, we had some relatively expensive `suiteSetup` and `suiteTeardown`, so even though our tests were passing `c8` would produce very incorrect reports, e.g. 40% as opposed to 70%. It took me a while to figure out what was going on given the nightmare that vscode unittesting tends to be.
To reproduce you can checkout our repo on commit https://github.com/fortran-lang/vscode-fortran-support/pull/629/commits/801c40ce4a80af7452931efda1b3f15e2f271ade and
```sh
npm ci
npm run coverage
```
An example where coverage is very incorrect is the file `linter-provider.ts`
Contributor guide
Assessment
This issue has not been assessed yet.