codeceptjs / codeceptjs/CodeceptJS

[bug] Retry + assert in async scenario does not work

オープン
#2,892 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
bug
主要言語
JavaScript
スター
4.2k
フォーク
757
平均マージ
2日 9時間
マージ済み PR(30日)
16

説明

#### What are you trying to achieve?

Assertion like the doc example (but with an unexpected value in the assertion):
```
const assert = require('assert')

Scenario.only('test page title 1', async ({ I }) => {
I.amOnPage('https://www.google.com')
const title = await I.grabTitle()
assert.strictEqual(title, 'Bad value')
}).retry(2)

Scenario('test page title 2', { retry: 2 }, async ({ I }) => {
I.amOnPage('https://www.google.com')
const title = await I.grabTitle()
assert.strictEqual(title, 'Bad value')
})
```

#### What do you get instead?

With retry option, the test does not fail (ok at the end), it's unexpected:

```
> codeceptjs run --steps "--verbose"

CodeceptJS v3.0.5
Using test root "---"
Helpers: Playwright, Context, Network
Plugins: screenshotOnFail, retryFailedStep, tryTo

US checkout / LP0001 --
[1] Starting recording promises
› [Session] Starting singleton browser session
test page title 1
I am on page "https://www.google.com"
I grab title
[1] Error | AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected

+ 'Google'
- 'Bad value'
[1] Starting session
[1] Stopping recording promises
› Test failed, try to save a screenshot
› Screenshot is saving to /----/codecept/output/test_page_title_1.failed.png
[2] Starting recording promises
› [Session] cleaning cookies and localStorage

OK | 0 passed // 2s
```

Without the retry, it's good, the test fails as expected:

```
> lv-front@1.0.2 codecept
> codeceptjs run --steps

CodeceptJS v3.0.5
Using test root "-----"

US checkout / LP0001 --
test page title 1
I am on page "https://www.google.com"
I grab title
✖ FAILED in 1059ms

-- FAILURES:

1) US checkout / LP0001
test page title 1:

Expected values to be strictly equal:
+ actual - expected

+ 'Google'
- 'Bad value'
+ expected - actual

-Google
+Bad value

AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected

Artifacts:
- screenshot: /----/codecept/output/test_page_title_1.failed.png

FAIL | 0 passed, 1 failed // 1s
```

### Details

* CodeceptJS version: 3.0.5 (but 3.0.6 tested too with the same bug)
* NodeJS Version: 14.16.1
* Operating System: macosX
* playwright

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。