bcoe / bcoe/c8

c8 + vitest: Some suites get covered other suites ...dont?

Open
#545 4 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
JavaScript
Stars
2.1k
Forks
99
PR merge metrics
No merged PRs in 30d

Description

## c8 + vitest: Some suites get covered other suites ...dont?

I'm working on modernizing and open-sourcing some platform...
It's test strategy and test suite is built of two layers:
* `test/e2e` - end-to-end suites that uses `child_process.exec` to run the SUT as cli in multiple scenarios.
* `test/unit` - unit-level suites that add unit-tests to cover specific code-paths which are not covered by any of the e2e scenarios.

In the old setup I used `nyc`, but after modernizing it to recent version of node and refactoring from `require` to `import` I could not get `nyc` to produce coverage for the `e2e` suites. Then I found about `c8`.

Astonishingly, `c8` produced coverage for the `e2e` right out of the box with zero configuration.
However, evidently, I was too fast to rejoice: It does not produce coverage for the unit tests... 🫨

The opened codebase is still young and does not take long to install or run - so I hope it can be used as a reproduction scenario...

## Reproduction

0. Clone `https://gitlab.com/prunjs/prun-core`, use branch `c8-bug-report`.

1. install (`npm i`)
2. "vanilla" run - `npm -w ecosystem/prun run cover`
![image](https://github.com/user-attachments/assets/0d2647f9-141b-44a2-a2a4-14f2b69752d6)
- Note that 3 suites have run: 1 e2e and 2 unit tests.
3. run only the e2e suites: `npm -w ecosystem/prun run cover -- test/e2e`
- *expect:* coverage numbers to be lesser than the vanilla run
- *found:* coverage are exactly like the "vanilla" run (i.e the "vanilla" run gathers coverage only from the e2e part)
5. run only unit tests - `npm -w ecosystem/prun run cover -- test/unit`
- *expect:* _some_ coverage to be produced
- *found*: The only covered file is `vitest.config.js`... no coverage is reported for any of the project files
![image](https://github.com/user-attachments/assets/18a3c2d9-4679-4353-8a75-bd8ae7a42d82)

I believe I could do a mish-mash where I ask both tools to produce coverage report in json for the parts it covers successfully, and ask either of the tools to merge them, but that would be a patch. I thought to ask here before I embark on something weird like that...

I hope it's just something silly I'm missing... donno...

🙏 thanks in advance 🙏

* **Version**:
```
10.1.2
```
* **Platform**:
Reproduced in both
* `fedora in WSL2` , with node `v20.17.0`, npm `v10.8.2'
* `Win11 pro`, with node `v20.12.2`, npm `v10.5.0`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.