karma-runner / karma-runner/karma-coverage

Karma coverage exit code depends on the order of the reporters array

Open
#471 0 comments 3 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
771
Forks
242
PR merge metrics
No merged PRs in 30d

Description

I'm using karma and karma-coverage in a CI context, failing the CI job when a test fails or coverage is too low. I get this information from the exit code returned by karma.

However, I noticed that karma doesn't return an error for low coverage IF the `text-summary` reporter is placed before the `html` reporter.

I've made a small repo showcasing this: https://github.com/mcdenhoed/karma-test

Is this expected? I didn't find any documentation indicating that the order of the reporters array was important. I've fixed my own use-case by just changing my `karma.conf.js` to have

```js
reporters: [
{type: "html"},
{type: "text-summary"},
]
```

instead of

```js
reporters: [
{type: "text-summary"},
{type: "html"},
]
```

This doesn't seem to me like intended behavior, but if it is, a note in the docs about it would be nice.

Contributor guide

Open the contributing guide

Research direction

Start with the linked reproduction repository and compare the two reporters arrays in karma.conf.js. Run its CI scenario with each reporter order and trace how the exit code is produced when coverage is too low. Done means determining whether reporter order should affect the exit code and adding a regression test or documenting the required order.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.