karma-runner / karma-runner/karma-coverage
Karma coverage exit code depends on the order of the reporters array
- Dominant language
- JavaScript
- Stars
- 771
- Forks
- 242
- PR merge metrics
- No merged PRs in 30d
Description
I'm using karma and karma-coverage in a CI context, failing the CI job when a test fails or coverage is too low. I get this information from the exit code returned by karma.
However, I noticed that karma doesn't return an error for low coverage IF the `text-summary` reporter is placed before the `html` reporter.
I've made a small repo showcasing this: https://github.com/mcdenhoed/karma-test
Is this expected? I didn't find any documentation indicating that the order of the reporters array was important. I've fixed my own use-case by just changing my `karma.conf.js` to have
```js
reporters: [
{type: "html"},
{type: "text-summary"},
]
```
instead of
```js
reporters: [
{type: "text-summary"},
{type: "html"},
]
```
This doesn't seem to me like intended behavior, but if it is, a note in the docs about it would be nice.
Contributor guide
Research direction
Start with the linked reproduction repository and compare the two reporters arrays in karma.conf.js. Run its CI scenario with each reporter order and trace how the exit code is produced when coverage is too low. Done means determining whether reporter order should affect the exit code and adding a regression test or documenting the required order.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100