codeceptjs / codeceptjs/CodeceptJS
[bug] Retry + assert in async scenario does not work
- 主要言語
- JavaScript
- スター
- 4.2k
- フォーク
- 757
- 平均マージ
- 2日 9時間
- マージ済み PR(30日)
- 16
説明
#### What are you trying to achieve?
Assertion like the doc example (but with an unexpected value in the assertion):
```
const assert = require('assert')
Scenario.only('test page title 1', async ({ I }) => {
I.amOnPage('https://www.google.com')
const title = await I.grabTitle()
assert.strictEqual(title, 'Bad value')
}).retry(2)
Scenario('test page title 2', { retry: 2 }, async ({ I }) => {
I.amOnPage('https://www.google.com')
const title = await I.grabTitle()
assert.strictEqual(title, 'Bad value')
})
```
#### What do you get instead?
With retry option, the test does not fail (ok at the end), it's unexpected:
```
> codeceptjs run --steps "--verbose"
CodeceptJS v3.0.5
Using test root "---"
Helpers: Playwright, Context, Network
Plugins: screenshotOnFail, retryFailedStep, tryTo
US checkout / LP0001 --
[1] Starting recording promises
› [Session] Starting singleton browser session
test page title 1
I am on page "https://www.google.com"
I grab title
[1] Error | AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected
+ 'Google'
- 'Bad value'
[1] Starting session
[1] Stopping recording promises
› Test failed, try to save a screenshot
› Screenshot is saving to /----/codecept/output/test_page_title_1.failed.png
[2] Starting recording promises
› [Session] cleaning cookies and localStorage
OK | 0 passed // 2s
```
Without the retry, it's good, the test fails as expected:
```
> lv-front@1.0.2 codecept
> codeceptjs run --steps
CodeceptJS v3.0.5
Using test root "-----"
US checkout / LP0001 --
test page title 1
I am on page "https://www.google.com"
I grab title
✖ FAILED in 1059ms
-- FAILURES:
1) US checkout / LP0001
test page title 1:
Expected values to be strictly equal:
+ actual - expected
+ 'Google'
- 'Bad value'
+ expected - actual
-Google
+Bad value
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected
Artifacts:
- screenshot: /----/codecept/output/test_page_title_1.failed.png
FAIL | 0 passed, 1 failed // 1s
```
### Details
* CodeceptJS version: 3.0.5 (but 3.0.6 tested too with the same bug)
* NodeJS Version: 14.16.1
* Operating System: macosX
* playwright
コントリビューションガイド
評価
この issue はまだ評価されていません。