karma-runner / karma-runner/karma

Karma Code Coverage Exit Code is always 0 even when test coverage threshold is not met

Open
#3,810 1 comment 3 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
12k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

Hey there im looking out for some support,

Am working on a project and trying to make a script stop if the code Coverage don't meet the predetermined thresholds.

Just for testing reasons and to show the issue i set the threshold for statements to 100% in my karma.config.js

I would now expect the script that I run to fail/stop when it comes to the karma code coverage, but it just shows me an error that the thresholds are not met and then proceed with the next part of my script.

After doing some research I can see that the exit code of the code coverage is 0 no matter if it mets the predetermined threshold or not. I guess this should be 1 if the threshold is not met so it would fail, but it isnt.
Screenshot 2022-08-30 091413

Is there a workaround, configuration or any kind of solution for this problem?

Im working with "karma": "6.3.20" and karma-coverage": "2.2.0

Any Help is appreciated

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the behavior with the reported Karma 6.3.20 and karma-coverage 2.2.0 versions, using the threshold in karma.config.js. Trace how the coverage threshold failure is reported and how the script's exit status is set. Done means an unmet threshold produces a nonzero exit code so the following script step does not run.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.