karma-runner / karma-runner/karma

Question: Karma + SauceLabs: Are tests relaunched after a browser disconnect?

Open
#2,337 2 comments 2 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
12k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

### Expected behaviour

After I get a DISCONNECT from a browser inside a SauceLabs test, does the code for the test get relaunched. It _seems_ that it is not executed again and that causes my test suite to fail after a DISCONNECT
### Actual behaviour

After a DISCONNECT the test spec that was executing at the moment of the disconnect fails.
### Environment Details
- Karma version (output of `karma --version`):
Karma version: 1.1.2
- Relevant part of your `karma.config.js` file

// Keep getting "Disconnected, because no message in 10000 ms."
// via: https://github.com/karma-runner/karma/issues/598
// and via: https://support.saucelabs.com/customer/en/portal/articles/2440724-karma-disconnected-tests-particularly-with-safari
browserDisconnectTimeout: 10000, // default 2000
browserDisconnectTolerance: 1, // default 0
browserNoActivityTimeout: 4_60_1000, //default 10000
captureTimeout: 4_60_1000, //default 60000
// Continuous Integration mode
// if true, Karma captures browsers, runs the tests and exits
singleRun: true,
concurrency: 3,

SauceLabs video: https://saucelabs.com/beta/tests/7e89c2d133b34f88b600014c20c15323/watch
Interested section of logs :
29 08 2016 18:27:42.457:INFO [Safari 7.0.6 (Mac OS X 10.9.5)]: Connected on socket /#3LDWIr7wePiXHQQUAAAJ with id 56288280
...
29 08 2016 18:27:55.911:DEBUG [Safari 7.0.6 (Mac OS X 10.9.5)]: Disconnected during run, waiting 10000ms for reconnecting.
29 08 2016 18:27:56.357:DEBUG [karma]: A browser has connected on socket /#5_WRtelzgfW5-IlhAAAK
29 08 2016 18:27:56.504:DEBUG [Safari 7.0.6 (Mac OS X 10.9.5)]: Reconnected on /#5_WRtelzgfW5-IlhAAAK.
29 08 2016 18:28:09.888:DEBUG [middleware:source-files]: Requesting /base/src/pym-loader.js /

IE 10.0.0 (Windows 7 0.0.0): Executed 48 of 48 SUCCESS (8.199 secs / 7.955 secs)
IE 11.0.0 (Windows 8.1 0.0.0): Executed 48 of 48 SUCCESS (8.102 secs / 7.963 secs)
Chrome 32.0.1700 (Mac OS X 10.9.5): Executed 48 of 48 SUCCESS (8.374 secs / 8.14 secs)
Mobile Safari 7.0.0 (iOS 7.0.3): Executed 48 of 48 SUCCESS (8.92 secs / 8.783 secs)
Firefox 26.0.0 (Mac OS X 10.9.0): Executed 48 of 48 SUCCESS (11.28 secs / 11.062 secs)
Chrome 51.0.2704 (Mac OS X 10.11.6): Executed 48 of 48 SUCCESS (9.273 secs / 7.83 secs)
Chrome Mobile 39.0.0 (Android 5.1.0): Executed 48 of 48 SUCCESS (9.966 secs / 9.828 secs)
Firefox 47.0.0 (Mac OS X 10.11.0): Executed 48 of 48 SUCCESS (10.823 secs / 8.838 secs)
Safari 9.1.2 (Mac OS X 10.11.6): Executed 48 of 48 SUCCESS (7.367 secs / 8.025 secs)
Safari 7.0.6 (Mac OS X 10.9.5): Executed 48 of 48 (1 FAILED) (28.128 secs / 27.862 secs)
TOTAL: 1 FAILED, 479 SUCCESS

1) should load pym to the page and autoinit the parent
pym-loader load with requirejs
Error: Timeout - Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.
Expected 1 to equal 0.

Contributor guide

Open the contributing guide

Research direction

Start with the supplied karma.config.js settings and Karma's browser disconnect/reconnect handling. Compare the disconnect and reconnection log entries with the final failed spec to determine whether the in-progress test is relaunched. Done means the behavior is confirmed and, if incorrect, captured by a reproducible test or documented resolution.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.