karma-runner / karma-runner/karma
Different Coverage Numbers Based On Test Run Order
- Dominant language
- JavaScript
- Stars
- 12k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
I have some tests that are producing different coverage numbers based on the order in which the tests are run.
I have components `A` and `B`; `B` is a dependency of `A`. If I run the tests for `A` then the tests for `B`, I get the coverage number for `A` that I expect but not for `B`; I get a much lower coverage number for `B`. However, if I reverse the order I get the order I get the coverage numbers I expect for `A` and `B`.
I believe this is because `A` uses some of the code in `B` so if it runs first the coverage for `B` is based on how `A` is using it. When the tests for `B` is run the coverage has already been calculated. This makes to me and matches what I am seeing. I am just wondering if this behavior is correct.
Sorry for the lack of actual code but I can't post it for proprietary reasons. I hope my descriptions makes sense. Thanks.
Contributor guide
Research direction
No repository files, tests, or entry points are identified. Start by reproducing the coverage difference with the described A/B test-run orders, then trace how coverage is collected across dependent components. Done means determining whether order-dependent coverage is expected and documenting or fixing the behavior based on that finding.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100