codeceptjs / codeceptjs/CodeceptJS

[bug] Retry + assert in async scenario does not work

Đang mở
#2,892 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
bug
Ngôn ngữ chính
JavaScript
Star
4.2k
Fork
757
Merge trung bình
2 ngày 9 giờ
Pull request đã merge (30 ngày)
16

Mô tả

#### What are you trying to achieve?

Assertion like the doc example (but with an unexpected value in the assertion):
```
const assert = require('assert')

Scenario.only('test page title 1', async ({ I }) => {
I.amOnPage('https://www.google.com')
const title = await I.grabTitle()
assert.strictEqual(title, 'Bad value')
}).retry(2)

Scenario('test page title 2', { retry: 2 }, async ({ I }) => {
I.amOnPage('https://www.google.com')
const title = await I.grabTitle()
assert.strictEqual(title, 'Bad value')
})
```

#### What do you get instead?

With retry option, the test does not fail (ok at the end), it's unexpected:

```
> codeceptjs run --steps "--verbose"

CodeceptJS v3.0.5
Using test root "---"
Helpers: Playwright, Context, Network
Plugins: screenshotOnFail, retryFailedStep, tryTo

US checkout / LP0001 --
[1] Starting recording promises
› [Session] Starting singleton browser session
test page title 1
I am on page "https://www.google.com"
I grab title
[1] Error | AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected

+ 'Google'
- 'Bad value'
[1] Starting session
[1] Stopping recording promises
› Test failed, try to save a screenshot
› Screenshot is saving to /----/codecept/output/test_page_title_1.failed.png
[2] Starting recording promises
› [Session] cleaning cookies and localStorage

OK | 0 passed // 2s
```

Without the retry, it's good, the test fails as expected:

```
> lv-front@1.0.2 codecept
> codeceptjs run --steps

CodeceptJS v3.0.5
Using test root "-----"

US checkout / LP0001 --
test page title 1
I am on page "https://www.google.com"
I grab title
✖ FAILED in 1059ms

-- FAILURES:

1) US checkout / LP0001
test page title 1:

Expected values to be strictly equal:
+ actual - expected

+ 'Google'
- 'Bad value'
+ expected - actual

-Google
+Bad value

AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected

Artifacts:
- screenshot: /----/codecept/output/test_page_title_1.failed.png

FAIL | 0 passed, 1 failed // 1s
```

### Details

* CodeceptJS version: 3.0.5 (but 3.0.6 tested too with the same bug)
* NodeJS Version: 14.16.1
* Operating System: macosX
* playwright

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.