karma-runner / karma-runner/karma

Karma's onerror handler coupled with Zone.js can cascade ErrorEvents into tests from which they did not originate

Open
#2,794 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
12k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

### Expected behaviour

Each Jasmine `it()` test should be independent; e.g.:

```js
describe('some test suite', () => {
it('should fail', () => {
fail('hi there');
});
it('should pass', () => {
expect(1).toEqual(1);
});
});
```

Should cause the first test to fail and the second to pass.

### Actual behaviour

The second test case, `should pass`, fails from an `ErrorEvent` re-raised from within Zone.js. I have not distilled this yet to a MWE, but as I have a workaround, I am posting the issue, since it took me a long time to correlate the `ErrorEvent` to karma's `onerror` handler.

### Environment Details

- Chrome 59
- Karma 1.7
- Ionic 3 application under test, setup similar to http://roblouie.com/article/376/ionic-2-set-up-unit-testing-the-best-way/
- jasmine-co installed globally after `zone.js` require statements

### Steps to reproduce the behaviour

TBD - I believe it was simply the above environment with two spec.ts testing modules.

### Workaround

In `karma-test-shim.js`, which contains the Angular `testing.TestBed.initTestEnvironment()` call, put this line:

//Fix karma - window.onerror being overridden would propagate errors into the
//next test, causing a cascading failure situation. Error from "throw err" in
//karma/static/debug.js.
beforeEach(function() { window.onerror = null; });

Note that I am not sure what other side effects this will have on UI tests. In my case, which is purely logic testing at the moment, it has no negative side effects.

Contributor guide

Open the contributing guide

Research direction

Begin with the reported Chrome 59/Karma 1.7 setup and the karma-test-shim.js workaround, then inspect karma/static/debug.js around the reported `throw err` path and its interaction with window.onerror. Reproduce the two Jasmine tests and confirm that an error in one test no longer causes the following test to fail.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.