codeceptjs / codeceptjs/CodeceptJS

[bug] Retry + assert in async scenario does not work

Abierto
#2,892 0 comentarios 0 reacciones 0 asignados Ver en GitHub
bug
Lenguaje dominante
JavaScript
Estrellas
4.2k
Forks
757
Merge medio
2 d 9 h
PR fusionados (30 d)
16

Descripción

#### What are you trying to achieve?

Assertion like the doc example (but with an unexpected value in the assertion):
```
const assert = require('assert')

Scenario.only('test page title 1', async ({ I }) => {
I.amOnPage('https://www.google.com')
const title = await I.grabTitle()
assert.strictEqual(title, 'Bad value')
}).retry(2)

Scenario('test page title 2', { retry: 2 }, async ({ I }) => {
I.amOnPage('https://www.google.com')
const title = await I.grabTitle()
assert.strictEqual(title, 'Bad value')
})
```

#### What do you get instead?

With retry option, the test does not fail (ok at the end), it's unexpected:

```
> codeceptjs run --steps "--verbose"

CodeceptJS v3.0.5
Using test root "---"
Helpers: Playwright, Context, Network
Plugins: screenshotOnFail, retryFailedStep, tryTo

US checkout / LP0001 --
[1] Starting recording promises
› [Session] Starting singleton browser session
test page title 1
I am on page "https://www.google.com"
I grab title
[1] Error | AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected

+ 'Google'
- 'Bad value'
[1] Starting session
[1] Stopping recording promises
› Test failed, try to save a screenshot
› Screenshot is saving to /----/codecept/output/test_page_title_1.failed.png
[2] Starting recording promises
› [Session] cleaning cookies and localStorage

OK | 0 passed // 2s
```

Without the retry, it's good, the test fails as expected:

```
> lv-front@1.0.2 codecept
> codeceptjs run --steps

CodeceptJS v3.0.5
Using test root "-----"

US checkout / LP0001 --
test page title 1
I am on page "https://www.google.com"
I grab title
✖ FAILED in 1059ms

-- FAILURES:

1) US checkout / LP0001
test page title 1:

Expected values to be strictly equal:
+ actual - expected

+ 'Google'
- 'Bad value'
+ expected - actual

-Google
+Bad value

AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected

Artifacts:
- screenshot: /----/codecept/output/test_page_title_1.failed.png

FAIL | 0 passed, 1 failed // 1s
```

### Details

* CodeceptJS version: 3.0.5 (but 3.0.6 tested too with the same bug)
* NodeJS Version: 14.16.1
* Operating System: macosX
* playwright

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.