gemini-testing / gemini-testing/html-reporter
Invalid screenshot file in playwright test causes entire test run to fail with "Error: Empty file"
- Dominant language
- TypeScript
- Stars
- 57
- Forks
- 69
- Avg merge
- 6d 22h
- Merged PRs (30d)
- 9
Description
### Verify latest release
- [x] I verified that the issue exists in the latest html-reporter release
### Html-reporter version
11.0.2
### Last html-reporter version that worked
none
### Link to the code that reproduces this issue or a replay of the bug
_No response_
### Reproduction steps
1. Set up Playwright (`npm init playwright`)
2. Add our reporter to the config:
```javascript
reporter: [
['html-reporter/playwright', {
enabled: true,
defaultView: 'failed',
path: 'html-report',
}],
],
```
3. Write a simple screenshot test:
```javascript
test('get screen', async ({ page }) => {
await page.goto('https://playwright.dev/');
await expect(page).toHaveScreenshot('landing-page.png');
});
```
4. Open the UI (`npx html-reporter gui --tool playwright`) and accept the screenshot after running the test
5. Replace the reference screenshot with a corrupted one (you can take any file and change its extension to `.png`)
6. Run the test, which will expectedly fail with the aforementioned error
### Actual Behavior
When an invalid screenshot is used as a reference image in a Playwright screenshot test, running such a test causes the entire Playwright test run to crash with an `Error: Empty file` error.
Full stack trace
```
Error: Empty file
at readFileSync (.../node_modules/.pnpm/image-size@1.1.1/node_modules/image-size/dist/index.js:77:19)
at imageSize (.../node_modules/.pnpm/image-size@1.1.1/node_modules/image-size/dist/index.js:112:23)
at getImageData (.../node_modules/.pnpm/html-reporter@10.18.7_playwright@1.46.1/node_modules/html-reporter/lib/adapters/test-result/playwright.ts:152:47)
at Array.map ()
at .../node_modules/.pnpm/html-reporter@10.18.7_playwright@1.46.1/node_modules/html-reporter/lib/adapters/test-result/playwright.ts:253:110
at Array.map ()
at PlaywrightTestResultAdapter.get imagesInfo [as imagesInfo] (.../node_modules/.pnpm/html-reporter@10.18.7_playwright@1.46.1/node_modules/html-reporter/lib/adapters/test-result/playwright.ts:248:69)
at baseGet (.../node_modules/.pnpm/lodash@4.17.21/node_modules/lodash/lodash.js:3077:24)
at basePickBy (.../node_modules/.pnpm/lodash@4.17.21/node_modules/lodash/lodash.js:3823:21)
at basePick (.../node_modules/.pnpm/lodash@4.17.21/node_modules/lodash/lodash.js:3802:14)
at Function. (.../node_modules/.pnpm/lodash@4.17.21/node_modules/lodash/lodash.js:13628:36)
at apply (.../node_modules/.pnpm/lodash@4.17.21/node_modules/lodash/lodash.js:489:27)
at Function.pick (.../node_modules/.pnpm/lodash@4.17.21/node_modules/lodash/lodash.js:6627:16)
at copyAndUpdate (.../node_modules/.pnpm/html-reporter@10.18.7_playwright@1.46.1/node_modules/html-reporter/lib/adapters/test-result/utils/index.ts:47:46)
at StaticReportBuilder.provideAttempt (.../node_modules/.pnpm/html-reporter@10.18.7_playwright@1.46.1/node_modules/html-reporter/lib/report-builder/static.ts:93:44)
at StaticReportBuilder.addTestResult (.../node_modules/.pnpm/html-reporter@10.18.7_playwright@1.46.1/node_modules/html-reporter/lib/report-builder/static.ts:121:38)
at .../node_modules/.pnpm/html-reporter@10.18.7_playwright@1.46.1/node_modules/html-reporter/playwright.ts:75:39
at processTicksAndRejections (node:internal/process/task_queues:105:5)
at run (.../node_modules/.pnpm/p-queue@5.0.0/node_modules/p-queue/index.js:211:14)
```
This behavior is already handled for testplane tests, but only occurs with Playwright.
### Expected Behavior
The main code that needs to be modified is the [adapter](https://github.com/gemini-testing/html-reporter/blob/master/lib/adapters/test-result/playwright.ts#L202). In the `imagesInfo` method, we need to add error handling for `ErrorName.INVALID_REF_IMAGE` (an example can be found in the testplane adapter).
The error type can only be matched by the text when it fails: "could not decode image as PNG"
### Definition of Done
- When an invalid screenshot is present in a test, other tests run correctly
- The report for the failed test shows an error about the invalid screenshot
### Which Node.js version are you using?
22
*Internal issue: https://nda.ya.ru/t/2rT1286B7JkvFj*
Contributor guide
Research direction
Start in lib/adapters/test-result/playwright.ts at the imagesInfo method referenced in the issue, then compare the existing invalid-reference handling in the testplane adapter. Confirm how the invalid PNG error text is surfaced and verify that other tests continue running while the failed report records the invalid screenshot error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- playwright, typescript
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 54/100