karma-runner / karma-runner/karma
Using browserDisconnectTolerance invalidates test suite
- Dominant language
- JavaScript
- Stars
- 12k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
### Expected behaviour
When a test fails, karma says that the test failed.
### Actual behaviour
When a test fails and chrome disconnects then reconnects (which is allowed if the `browserDisconnectTolerance` is set to something greater than 0), karma will only report the results of the tests that have run since reconnecting, ignoring any test failures from before chrome disconnected.
### Environment Details
- Karma version (output of `karma --version`): 1.7.1
- Relevant part of your `karma.config.js` file:
```
browserDisconnectTolerance: 1
```
### Steps to reproduce the behaviour
1. Set `browserDisconnectTolerance` to anything greater than 0.
2. Run a test that fails before the browser disconnects, then after reconnecting run a test that passes
3. _Expected_: The build should fail and karma should report the test failure from before the disconnection
_Actual_: The build passes because karma only reports the results of tests run since the reconnection
Contributor guide
Research direction
Start with the reproduction described in karma.config.js using browserDisconnectTolerance greater than 0: run a failing test before Chrome disconnects and a passing test after it reconnects. Trace how Karma collects and reports results across the reconnect, then verify that the earlier failure still makes the build fail and is reported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100