codeceptjs / codeceptjs/CodeceptJS

[bug] Retry + assert in async scenario does not work

Offen
#2,892 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug
Vorherrschende Sprache
JavaScript
Sterne
4.2k
Forks
757
Ø Merge
2 T. 9 Std.
Gemergte PRs (30 T.)
16

Beschreibung

#### What are you trying to achieve?

Assertion like the doc example (but with an unexpected value in the assertion):
```
const assert = require('assert')

Scenario.only('test page title 1', async ({ I }) => {
I.amOnPage('https://www.google.com')
const title = await I.grabTitle()
assert.strictEqual(title, 'Bad value')
}).retry(2)

Scenario('test page title 2', { retry: 2 }, async ({ I }) => {
I.amOnPage('https://www.google.com')
const title = await I.grabTitle()
assert.strictEqual(title, 'Bad value')
})
```

#### What do you get instead?

With retry option, the test does not fail (ok at the end), it's unexpected:

```
> codeceptjs run --steps "--verbose"

CodeceptJS v3.0.5
Using test root "---"
Helpers: Playwright, Context, Network
Plugins: screenshotOnFail, retryFailedStep, tryTo

US checkout / LP0001 --
[1] Starting recording promises
› [Session] Starting singleton browser session
test page title 1
I am on page "https://www.google.com"
I grab title
[1] Error | AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected

+ 'Google'
- 'Bad value'
[1] Starting session
[1] Stopping recording promises
› Test failed, try to save a screenshot
› Screenshot is saving to /----/codecept/output/test_page_title_1.failed.png
[2] Starting recording promises
› [Session] cleaning cookies and localStorage

OK | 0 passed // 2s
```

Without the retry, it's good, the test fails as expected:

```
> lv-front@1.0.2 codecept
> codeceptjs run --steps

CodeceptJS v3.0.5
Using test root "-----"

US checkout / LP0001 --
test page title 1
I am on page "https://www.google.com"
I grab title
✖ FAILED in 1059ms

-- FAILURES:

1) US checkout / LP0001
test page title 1:

Expected values to be strictly equal:
+ actual - expected

+ 'Google'
- 'Bad value'
+ expected - actual

-Google
+Bad value

AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected

Artifacts:
- screenshot: /----/codecept/output/test_page_title_1.failed.png

FAIL | 0 passed, 1 failed // 1s
```

### Details

* CodeceptJS version: 3.0.5 (but 3.0.6 tested too with the same bug)
* NodeJS Version: 14.16.1
* Operating System: macosX
* playwright

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.