bcoe / bcoe/c8

Coverage information too high for large esbuild produced bundle

Open
#502 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
2.1k
Forks
99
PR merge metrics
No merged PRs in 30d

Description

* **Version**: v20.9.0
* **Platform**: Linux JABAILE-DESK02 5.15.90.1-microsoft-standard-WSL2 #1 SMP Fri Jan 27 02:56:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

I can only apologize in advance for this one...

I work on [TypeScript](http://github.com/microsoft/TypeScript), and I've been wanting to set up coverage information for the compiler. c8 does a great job; significantly faster than nyc (of course). I use c8 in my own projects with success.

However, capturing coverage for the compiler produces results that are too high to believe. `checker.ts` is a 50k line file, and yet its coverage is:

```
checker.ts | 99.99 | 95.38 | 98.62 | 99.99 | 50577-50578
```

There's no way that we actually have tested every line of the checker. And indeed, adding a function to a known-dead path (like one ending in `Debug.assertNever`) indicates that the line is covered when it definitely is not, and the overall count doesn't change.

![image](https://github.com/bcoe/c8/assets/5341706/9183aa93-4c10-49ad-a326-b80c5c804088)

I don't know if I have a good repro for this besides `npx c8 npm test -- --no-lint` on our repo. Obviously that's huge but I don't really know where to start.

At best, I can observe that `npx c8 npm test -- --no-lint --no-bundle` works better, i.e. not using esbuild and instead running the tsc output produces a more believable coverage count:

```
checker.ts | 97.5 | 95.87 | 98.62 | 97.5 | ...-48684,48686-48687,48761-48762,48 ...
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.