karma-runner / karma-runner/karma
Karma's onerror handler coupled with Zone.js can cascade ErrorEvents into tests from which they did not originate
- Dominant language
- JavaScript
- Stars
- 12k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
### Expected behaviour
Each Jasmine `it()` test should be independent; e.g.:
```js
describe('some test suite', () => {
it('should fail', () => {
fail('hi there');
});
it('should pass', () => {
expect(1).toEqual(1);
});
});
```
Should cause the first test to fail and the second to pass.
### Actual behaviour
The second test case, `should pass`, fails from an `ErrorEvent` re-raised from within Zone.js. I have not distilled this yet to a MWE, but as I have a workaround, I am posting the issue, since it took me a long time to correlate the `ErrorEvent` to karma's `onerror` handler.
### Environment Details
- Chrome 59
- Karma 1.7
- Ionic 3 application under test, setup similar to http://roblouie.com/article/376/ionic-2-set-up-unit-testing-the-best-way/
- jasmine-co installed globally after `zone.js` require statements
### Steps to reproduce the behaviour
TBD - I believe it was simply the above environment with two spec.ts testing modules.
### Workaround
In `karma-test-shim.js`, which contains the Angular `testing.TestBed.initTestEnvironment()` call, put this line:
//Fix karma - window.onerror being overridden would propagate errors into the
//next test, causing a cascading failure situation. Error from "throw err" in
//karma/static/debug.js.
beforeEach(function() { window.onerror = null; });
Note that I am not sure what other side effects this will have on UI tests. In my case, which is purely logic testing at the moment, it has no negative side effects.
Contributor guide
Research direction
Begin with the reported Chrome 59/Karma 1.7 setup and the karma-test-shim.js workaround, then inspect karma/static/debug.js around the reported `throw err` path and its interaction with window.onerror. Reproduce the two Jasmine tests and confirm that an error in one test no longer causes the following test to fail.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100