gemini-testing / gemini-testing/html-reporter
Playwright: Accept uses the GUI host platform for snapshots from Linux CI
- Dominant language
- TypeScript
- Stars
- 57
- Forks
- 69
- Avg merge
- 6d 22h
- Merged PRs (30d)
- 9
Description
I'm trying to accept screenshots from Linux CI on my Mac, without rerunning the tests locally. CI already has the images I want to commit, but Accept writes `example-darwin.png` instead of updating `example-linux.png`.
[`getScreenshotPath`](https://github.com/gemini-testing/html-reporter/blob/a4e797c/lib/adapters/config/playwright.ts) uses the GUI host's `process.platform` for `{platform}` and the default `{snapshotSuffix}`.
### Reproduction
With Playwright 1.59.1 and a template like:
```ts
snapshotPathTemplate: '{testDir}/__screenshots__/{arg}-{platform}{ext}'
```
1. On Linux, generate a baseline for `expect(page).toHaveScreenshot('example.png')` using `npx playwright test --update-snapshots`.
2. Change the page appearance and run `npx playwright test --reporter=blob` on Linux.
3. Download the blob into a macOS checkout of the same revision, keeping the original Linux baseline.
4. Run `npx playwright merge-reports --reporter=html-reporter/playwright ./blob-report`, then `npx html-reporter gui --tool playwright`.
5. Click Accept and check the changed files. The target should be the Linux baseline, not a new Darwin file. Undo should restore that same file.
I investigated this on a 11.16.1-based fork with Node 20.19.6. The path resolver still produces the Darwin filename on upstream 11.16.2; I haven't rerun the full GUI flow on that release.
Small resolver-only repro on macOS
Run from an installed source checkout of html-reporter:
```sh
node -r ./test/setup/ts-node <<'NODE'
const path = require('node:path');
const {PlaywrightConfigAdapter} = require('./lib/adapters/config/playwright.ts');
const config = {
testDir: process.cwd(),
snapshotPathTemplate: '{testDir}/__screenshots__/{arg}-{platform}{ext}',
projects: []
};
const test = {file: 'visual.spec.ts', browserId: 'chromium', titlePath: ['visual']};
console.log(path.basename(new PlaywrightConfigAdapter(config).getScreenshotPath(test, 'example')));
NODE
```
Prints `example-darwin.png`: the resolver has no input for the source run's platform.
### Possible fix
I have a [prototype](https://github.com/dimazollo/html-reporter/pull/1) that saves the source platform per result. Direct runs use their process platform; blob merges read `onBlobReportMetadata.params.userAgent` through the internal V2 `onReportConfigure` hook. Accept/Undo worked with a Linux report on macOS even after removing the blobs, and Undo restored the original checksum.
I'd like to avoid requiring a platform flag for every downloaded report. The concern with this implementation is its reliance on private Playwright hooks and user-agent parsing. Older blobs need a fallback, and arbitrary custom `snapshotSuffix` values aren't recovered. The prototype also contains a separate error-classification fix, which I would leave out of the platform PR.
Would you accept per-result platform metadata, or prefer preserving the resolved snapshot path? Is there a supported way to get this during blob merging that I've missed? I'd appreciate guidance on that and on the fallback for old reports before preparing the PR.
Contributor guide
Research direction
Start in lib/adapters/config/playwright.ts, especially getScreenshotPath, and run the resolver-only reproduction from the issue. Then trace the blob merge and Accept/Undo flow described in the reproduction, including the prototype's per-result platform metadata. Done means a Linux blob accepted from macOS updates example-linux.png and Undo restores that file, with an explicit fallback for older reports.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- playwright, typescript
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100