karma-runner / karma-runner/karma-coverage

Coverage timeouts and performance problems

Open
#221 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
771
Forks
242
PR merge metrics
No merged PRs in 30d

Description

I'm a maintainer on the [Cesium](https://github.com/AnalyticalGraphicsInc/cesium) project and we've recently switched our 200,000 line JavaScript code base over to use karma (over 7200 jasmine unit tests). Besides an issue with binary files that we submitted a [PR](https://github.com/karma-runner/karma/pull/1816) for (which has since been merged). everything went off without a hitch and works phenomenally well. The entire test suite runs in less than a minute. Thank you for such a great project.

That being said, we are now trying to switch over code coverage from an ancient version of JSCoverage to istanbul via karma-coverage and running into a major performance hurdle. Our system takes about 90 seconds total to instrument and cover. However, running the same tests via karma-coverage goes on for several minutes and ultimately fails. I had to jack up browserNoActivityTimeout and browserDisconnectTimeout just to try and get it to run to completion (and that has only happened once so far and took 10 minutes, the results are not repeatable). A few seemingly innocuous tests seem to take multiple minutes to complete on their own (but execute in milliseconds when not run under coverage). I'm not sure it's any particular test causing issues though as commenting out "problem" tests still cause the whole thing to run slowly.

I've tried multiple launchers and browsers in case that was the problem (we usually use Chrome by default), but all browsers exhibited the same performance issue.

While I don't expect anyone to magically solve my problem, I am hoping that someone can provide tips for tracking down whatever the root cause of the problem may be. Perhaps there are some options I can tune or extra logging commands I can run to figure out what is taking so long.

Here's the relevant configuration we are using:

https://github.com/AnalyticalGraphicsInc/cesium/compare/coverage
https://github.com/AnalyticalGraphicsInc/cesium/blob/coverage/Specs/karma-main.js
https://github.com/AnalyticalGraphicsInc/cesium/blob/coverage/Specs/karma.conf.js

Any help would be greatly appreciated, and thanks again for a great project.

As a full disclaimer, I asked this question on the mailing list 3 weeks ago and received 0 replies, I'm hoping that I was just asking in the wrong place and that posting an issue here is okay. Thanks again.

Contributor guide

Open the contributing guide

Research direction

Start with the linked Cesium coverage configuration, especially Specs/karma-main.js and Specs/karma.conf.js, and reproduce the suite with karma-coverage. Compare coverage and non-coverage timings while using the available timeout and logging options to identify whether instrumentation, particular tests, or browser execution causes the slowdown. Done means identifying a reproducible bottleneck or a configuration-level cause.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
performance, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.