WordPress / WordPress/contributor-toolkit

Screenshot harness is still not reproducible across a full run

Open
#326 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
36
Forks
13
Avg merge
23h 19m
Merged PRs (30d)
72

Description

A full npm run shots run is still not always reproducible. site-menu.png and stale-site-notice.png sometimes differ between two runs with no code change in between.

Each of those shots is reproducible on its own under --only. That rules out the shot's own prepare and points at state shared across shots in one app instance — the fixture tier launches one Electron per variant and reloads the window between shots, which resets the renderer but not the main process, its network stack, or any cached GitHub response.

Two causes were already fixed in #277: CSS animations are frozen at their first frame, and selectSite waits out the linked-pull-request lookup. Those took the churn from three images per run to an occasional two, and fixed a real docs bug on the way — the same run had produced site-view.png mid-check and trac-ticket-panel.png already resolved.

Ending it properly means the fixture never reaching GitHub: a test-mode flag the main process reads, serving canned responses instead. That is app-level work, not harness work, which is why it is not in #277.

Worth doing because the UI polish pass is a stack of seven PRs over these same images. A PNG that changes on its own turns every rebase into a binary conflict on a file nothing actually changed, and a binary conflict has no resolution but to pick a side and re-capture.

To reproduce: npm run shots, git checkout docs/, npm run shots again, git status.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the fixture used by npm run shots and the Electron main-process entry point that handles GitHub or network responses; review the related changes in #277. Reproduce with npm run shots, then verify that repeated full runs, including after git checkout docs/, leave the screenshots unchanged and git status clean.

Written by the indexing model from the issue text.

Assessment

Tech stack
electron, javascript
Domain
desktop, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.