codeceptjs / codeceptjs/CodeceptJS

Running tests with workers causing incorrect failure / passing count

Abierto
#2,158 11 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
JavaScript
Estrellas
4.2k
Forks
757
Merge medio
2 d 9 h
PR fusionados (30 d)
16

Descripción

#### What are you trying to achieve?
To run a series of test features expect 0 or positive failures, or 100% success across all 78 tests.

#### What do you get instead?
In the output below, one test failed but the output says all test have passed (although there are 78 total, the count says 77.)
```
[5] ✔ [testname]
[2] ✔ [testname]
[8] ✖ [testname] 264128ms
[8] [featurename] --
[8] ✔ [testname]
[5] ✔ [testname]
[2] ✔ [testname]
[2] ✔ [testname]
[5] ✔ [testname]
[5] ✔ [testname]
[2] ✔ [testname]
[5] ✔ [testname]
[2] ✔ [testname]
[5] ✔ [testname]
[5] ✔ [testname]
[5] ✔ [testname]
[5] ✔ [testname]
[5] ✔ [testname]
[5] ✔ [testname]

OK | 77 passed // 1106.326s
/tmp/build/3e139e9d
+ '[' 0 -ne 0 ']'
```

In other instances, with no pipeline changes, it would say all tests passed with -1 failed.
```
FAIL | 78 passed, -1 failed // 1144.407s
```

Codecept is being run with:
```
codeceptjs run-workers --suites 8 --grep @full-suite
```

These asynchronous tests were passing OK for some time. They are all being ran within a codecept container.

### Details

* CodeceptJS version: v2.4.1
* NodeJS Version: v12.10.0
* Operating System: Not sure
* puppeteer || webdriverio || protractor || testcafe version (if related) Not sure
* Configuration file:

```js
exports.config = {
tests: './tests/*.js',
timeout: 10000,
output: './output',
helpers: {
Puppeteer: {
chrome: { args: ['--no-sandbox'] },
url: 'http://localhost:8000',
show: false,
restart: false,
keepBrowserState: true,
windowSize: '1200x800',
waitForNavigation: 'networkidle0',
waitForTimeout: 30000,
},
DomHelpers: { require: './helpers/DomHelpers.js' },
},
include: {
I: './steps/steps_file.js',
[more stuff]
},
plugins: {
retryFailedStep: {
enabled: true,
retries: 5,
minTimeout: 500,
},
[customplugin]
},
bootstrap: false,
// bail will fail hard after one test failure
mocha: { bail: false },
name: '[name]',
};
```

Hopefully the info I've provided is sufficient -- please let me know if I can provide anything else!

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.