codeceptjs / codeceptjs/CodeceptJS
Running tests with workers causing incorrect failure / passing count
- 主要语言
- JavaScript
- 星标
- 4.2k
- 派生
- 757
- 平均合并
- 2 天 9 小时
- 30 天内合并 PR
- 16
描述
#### What are you trying to achieve?
To run a series of test features expect 0 or positive failures, or 100% success across all 78 tests.
#### What do you get instead?
In the output below, one test failed but the output says all test have passed (although there are 78 total, the count says 77.)
```
[5] ✔ [testname]
[2] ✔ [testname]
[8] ✖ [testname] 264128ms
[8] [featurename] --
[8] ✔ [testname]
[5] ✔ [testname]
[2] ✔ [testname]
[2] ✔ [testname]
[5] ✔ [testname]
[5] ✔ [testname]
[2] ✔ [testname]
[5] ✔ [testname]
[2] ✔ [testname]
[5] ✔ [testname]
[5] ✔ [testname]
[5] ✔ [testname]
[5] ✔ [testname]
[5] ✔ [testname]
[5] ✔ [testname]
OK | 77 passed // 1106.326s
/tmp/build/3e139e9d
+ '[' 0 -ne 0 ']'
```
In other instances, with no pipeline changes, it would say all tests passed with -1 failed.
```
FAIL | 78 passed, -1 failed // 1144.407s
```
Codecept is being run with:
```
codeceptjs run-workers --suites 8 --grep @full-suite
```
These asynchronous tests were passing OK for some time. They are all being ran within a codecept container.
### Details
* CodeceptJS version: v2.4.1
* NodeJS Version: v12.10.0
* Operating System: Not sure
* puppeteer || webdriverio || protractor || testcafe version (if related) Not sure
* Configuration file:
```js
exports.config = {
tests: './tests/*.js',
timeout: 10000,
output: './output',
helpers: {
Puppeteer: {
chrome: { args: ['--no-sandbox'] },
url: 'http://localhost:8000',
show: false,
restart: false,
keepBrowserState: true,
windowSize: '1200x800',
waitForNavigation: 'networkidle0',
waitForTimeout: 30000,
},
DomHelpers: { require: './helpers/DomHelpers.js' },
},
include: {
I: './steps/steps_file.js',
[more stuff]
},
plugins: {
retryFailedStep: {
enabled: true,
retries: 5,
minTimeout: 500,
},
[customplugin]
},
bootstrap: false,
// bail will fail hard after one test failure
mocha: { bail: false },
name: '[name]',
};
```
Hopefully the info I've provided is sufficient -- please let me know if I can provide anything else!
贡献指南
评估
这个 Issue 还没有评估数据。