maniator / maniator/verticopolis

[Bug]: [P3] The visual-regression baselines launch Chromium without the deterministic rasterization flags

Open
#797 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
5
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Raised while fixing #762, and deliberately left out of that PR.

#762 traced an intermittent screenshot-determinism failure to Chromium being launched with no arguments, so every rasterization decision came from runtime detection and could resolve differently between two instances competing in one container. The evidence was 1,456 pixels differing only in antialiasing coverage, at identical glyph positions. The fix passes five deterministic flags to the launch in scripts/screenshots.ts.

playwright.config.ts's launchOptions, which drive e2e/visual.spec.ts and its committed *-snapshots baselines, are the same class of exposure and did not get the same treatment.

Why it was not bundled into the #762 fix

Those baselines are committed pixels re-minted only through update-visual-baselines.yml (the [update-baselines] marker), which is a different flow from the drift check's commit-on-approval job. Putting two independent pixel refreshes through two different mechanisms in one PR would have made both harder to review and harder to roll back.

Why it is P3 rather than urgent

The visual baselines are not currently observed flaking. Worth knowing, though: retries is set on CI, so a flake of this class would be masked by a passing retry rather than reported. Absence of evidence here is weak evidence of absence.

Fix shape

Import the same reviewed flag list (DETERMINISTIC_CHROMIUM_ARGS in src/tests/screenshotLaunchArgs.ts) into playwright.config.ts's launchOptions.args, then re-mint the baselines through update-visual-baselines.yml in the same PR if the flags move any pixels. The #762 work measured four of the five flags as pixel-neutral on a host A/B, with --disable-lcd-text the only mover, and expects it to be a no-op on Linux since the container already renders grayscale AA. So a refresh may not even be needed, but the baseline flow is the arbiter.

The guard test added in the #762 fix (src/tests/screenshotLaunchArgs.guard.test.ts) pins the inventory of Chromium launchers under scripts/ and its exemption list, so this file is a declared exemption rather than an oversight. Closing this issue means moving it out of that list.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with playwright.config.ts and the reviewed flag list in src/tests/screenshotLaunchArgs.ts, then read src/tests/screenshotLaunchArgs.guard.test.ts and the update-visual-baselines.yml flow. Configure the visual tests to use the deterministic arguments, run the guard and visual suites, and re-mint committed baselines only if the flags change pixels; done means the file is removed from the guard test's exemption list.

Written by the indexing model from the issue text.

Assessment

Tech stack
playwright, typescript
Domain
ci-cd, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.