Closing the testing loop: eleven proposals, with a working prototype for each

What problem does this address?

The toolkit handles the hard part very well. Getting a wordpress-develop environment running with no prerequisites is the thing nobody else has solved, and it works.

This proposal is about what happens after that. The app takes a contributor to "the patch is applied" and stops, and the steps after that are the ones that turn a session into a contribution.

I used the app for real work, kept a note of every point where I had to leave it or do something by hand, and then built each item as a working prototype against trunk. I have since used that prototype to file a genuine test report on Trac #20810.

They are independent. Any one stands on its own, and I would rather you picked from them than took all eleven.

#350 says the goal "is not to be used, it is to produce WordPress contributors." Measured against that, here is what is missing.

• The environment is a black box. #144 already names this: a contributor cannot answer "why don't I see my change?", so a change that does not appear looks like a failure rather than a missing precondition. The app knows every fact needed to answer it and shows none of them.
• The report is the contribution, and the app does not help write it. A tester who applies a patch, watches it work, and posts nothing has produced nothing. A test report is the most common first contribution there is, and assembling one by hand is most of the work of making it.
• Nothing is remembered. Since #108 a ticket is a branch rather than a site, which is the right shape and has a consequence: a ticket's record lives with the checkout that holds it, so nothing in the app can answer "what have I worked on" once the work spreads across more than one site.
• A finished report survives until the next copy. It lives on the clipboard and nowhere else.
• Right-click does nothing. Electron ships no context menu. That does not read as "this app has no menu", it reads as "right-click is broken", and what it costs first is paste, in the two places the app asks people to type real text.
• There is no way out to a real shell. The Terminal panel is a prompt over the app's own runner, so git log or a test run has nowhere to go. #350 calls the app "a silo with an easy way in and no way out".
• The running version is invisible, and there is no way to the newer one. The version appears only in the log file, so a contributor on an old build has no way to find out, and no route to the release that would fix what they are hitting.
• Two copies can silently lose each other's work. The site registry is the one thing in this app a contributor cannot reconstruct, and nothing currently stops a second instance overwriting it.

What is your proposed solution?

Where all of this sits

Before the detail, the whole site screen with everything in place, so the pieces below have somewhere to belong. Nothing existing was moved or removed.

And the same screen on a site with no ticket linked yet, so it is clear what is conditional and what is always there.

---

1. Say what the environment is

Every fact here was already in the app, spread across siteMeta, the ticket branch's record and the running server. None of it was on screen.

Serving answers #144's question most often. Playground mounts build/, so an edit in src/ is invisible until a build runs, and this says which of the two is live.

Multisite states plainly that the app cannot provide it, rather than staying silent. Playground serves on loopback and the generated wp-config carries no MULTISITE constant, so a patch gated behind it can never be reached here. Saying so costs a line and saves an afternoon.

Applied names the pull request and the commit it went on top of. "I tested PR #12846" means little a month later when trunk has moved.

The rows are conditional, which is why the two screenshots above differ: a site standing on trunk with no ticket shows neither Ticket nor Applied, rather than showing them empty.

One detail worth flagging, because I got it wrong first and caught it in review: the version is read from whichever tree is actually being served. The build stamps a date into the version, so src/ says 7.2-alpha-63166-src while the running site reports 7.2-alpha-20260820.014058. Quoting the source version under a line that says "Serving: build/" puts a string into a public test report that get_bloginfo('version') never returns on that site.

2. Help write the report

Most of a test report is the Environment block, which the app can now fill in on its own. What is left is the part only the contributor knows, in the order the handbook asks for it.

The wordpress.org handle comes from the preferences already collected for patch provenance, so it is not asked for twice.

Until the two questions that matter are answered, the button says so rather than failing on click:

3. Make "I could not test it" a real answer

Three verdicts, not two, and the third is offered as an equal rather than buried.

This is #144's argument made concrete. A change gated behind an unmet condition is not a failure, and a form offering only pass and fail forces the tester to report one. Multisite is the standing example, since this app cannot provide it at all.

Here is what that verdict actually produces, kept as a finished report:

The report says what was attempted and why it stopped, which is useful to a committer. "Fail" would not have been.

4. Show exactly what will be copied

The preview is the text that goes on the clipboard, in the markup the destination renders, with no second code path that could drift from it.

Nothing is posted. Following #107, the report goes to the clipboard and the ticket opens; pasting it is the contributor's act, because posting needs their wordpress.org session and "is a step a first-time contributor should learn anyway". The line under the button says exactly that.

The formatting buttons write the markup the destination renders: a ticket gets '''bold''', a pull request gets **bold**. Nobody remembers Trac's WikiFormatting, and one set of buttons that always wrote it would put literal apostrophes through a GitHub comment.

5. Remember what has been worked on

Because a ticket's record lives with the checkout that holds it, nothing could answer this across sites before.

The middle number is called out on its own because it is the count that is actually a contribution, and the gap between it and the rest is the thing worth noticing about your own habits.

It says only what the app witnessed. "Applied" never becomes "fixed" and "wrote up a report" never becomes "accepted", because the app cannot see what happened on Trac afterwards. A history that overstated itself would be worse than none, because somebody would rely on it when props are given out.

The bottom row points at a checkout that has since been deleted. The record of the work stays; only the button to open the site goes.

6. Keep the reports

Until now the only copy of a finished report lived on the clipboard until the next thing overwrote it. If the browser did not open, or the paste went wrong, or the ticket wanted a correction an hour later, the writing was simply gone.

Each is now kept with its exact text, in both formats, so it can be copied again or checked before replying to a committer who has questioned it.

Open it goes to the ticket the report was written for, which is where the conversation about it will be. Copy again puts the same text back on the clipboard, unchanged. Forget is the only thing that removes it, and nothing removes it on its own.

7. A right-click menu

Electron ships none. The menu is built from Chromium's own editFlags, so nothing is offered that the focused element cannot do: no Paste into a read-only log pane, and Copy greyed out when nothing is selected, as it is above.

The terminal needs its own, because xterm takes input through a hidden helper textarea and a right-click lands on the screen div, so Chromium reports a surface that plainly accepts typing as not editable. That is why the menu above also carries Select All and Clear, which are the terminal's own rather than Chromium's.

A detail found by running it rather than reasoning about it: Ctrl+V in the terminal pasted twice and came out as npm run buildld. Returning false stops xterm handling the key but not the browser performing its own paste, so it needs preventDefault() as well. A pasted newline is also flattened to a space, so a copied command lands at the prompt without running itself.

8. A way out to a real shell

The button sits in the terminal's own header, next to the thing it stands in for, and stays in the More menu for anyone who learned it there.

It opens carrying the panel's scrollback, printed above a live prompt, so the window opens on what was already on screen rather than an empty shell that has to be reconstructed from memory.

A rule marks where the copied history stops:

---- end of the history copied from the app ----
You are now in a real shell in this folder. What you type here is not sent back to the app.

The panel also gained padding, a drag handle and keyboard resize. Its text sat flush against the edge before, because xterm paints its rows from the first pixel of its container.

9. Say which build is running

It only appeared in the log file. The footer now carries it, so "which build are you on?" is answerable without asking somebody to find a log.

10. Offer the update, and be honest when the check fails

Four states, not two, because "I could not find out" is not the same as "you are up to date", and telling somebody the second when the first is true is how an app ends up trusted less than its own log file.

• An update exists. The heading names the version and the button carries the size of the file, so a download this large is not a surprise.
• You are up to date.
• You are ahead of the latest release, which is what a build made from source looks like, and is worth saying rather than leaving as a puzzle. That is the state in the screenshot.
• The check did not get an answer. A rate-limited request, an offline machine and a rewritten response are indistinguishable from here, and none of them is evidence about the version, so none of them is reported as one.

The download is offered only when the release actually carries a build for this machine. Assets are matched on the pieces electron-builder's artifactName puts in the file name rather than on a hardcoded list, so a release that adds an arm64 Windows build is found without this being edited. On macOS the architecture is not negotiable: an Intel machine handed the arm64 disk image gets a download that will not open, which is worse than being sent to the releases page to choose.

The app does not install it, and says so. The download opens in the browser and the line under the button explains that running it is the contributor's step. An app that downloads and then appears to do nothing reads as an app that failed.

Release notes are trimmed, not parsed. Whatever a maintainer wrote is what a contributor should read, and a summariser here would eventually disagree with the release page it claims to be quoting.

The footer only draws attention to itself when there is an actual update. Being current, being ahead and not knowing are all states where a badge would be asking for a click that leads nowhere.

11. One instance, so the registry cannot be lost

electron-store keeps its contents in memory and rewrites the whole file on every set, and settings-store.js constructs it once per process. Two running copies therefore each hold their own snapshot: register a site in one, save a report in the other, and the second write silently drops the first.

The site registry is the one thing in this app a contributor cannot reconstruct, so this is a data-loss bug waiting for somebody to double-click the icon twice.

The lock is taken before ready on purpose. A second copy has to find out it is second and quit before it has built a window or touched the store. Rather than quitting silently, which reads as the app being broken, it brings the existing window to the front and restores it if it was minimised.

It is not applied to a development build whose userData has been redirected, because the screenshot harness and the end-to-end tests deliberately run several copies at once against separate stores, and there is no shared file for them to fight over.

---

Smaller things in the same pass

• Stop everything, shown only while something is running. The server and the watch stay separately controllable (#247); this is the shortcut for wanting neither.
• The three site actions share one shape at WordPress's 2px radius. They were a 12px radius, a 10px radius, one primary and two secondary.
• The sidebar uses @wordpress/icons, the set wp-admin itself uses, which is the only thing that still identifies an entry once the sidebar is collapsed.
• Playground signs the admin in, so wp-admin opens ready rather than at a login form.

---

How it is built

Eleven pure modules, each with its own test file, following the standard in the review instructions: anything with more than one branch lives in a module rather than in index.jsx, because that file mounts itself at module scope and the suite cannot reach it.

Module	What it does
site-environment.cjs	What a site is. Performs no I/O, so the renderer can import what builds on it.
test-report.cjs	The report, in Trac WikiFormatting or GitHub Markdown.
work-history.cjs	The cross-site record.
report-store.cjs	Finished reports, kept.
context-menu.cjs	The right-click menu, built from Chromium's own editFlags.
terminal-launch.cjs	A real terminal, per platform, carrying the scrollback.
update-check.cjs	Version comparison semver-aware including prereleases, and the asset for this machine.
renderer/update-dialog.cjs	What the update window says, per state.
renderer/report-form.cjs	The form's shape and its submit state.
renderer/markup-toolbar.cjs	Formatting that follows the destination.
renderer/work-history-view.cjs	The strings the history screen shows.

New IPC channels are registered in test/ipc-wiring.test.cjs, so the coverage guard holds. 1215 tests pass and lint is clean.

Two decisions the review pass changed, worth naming because they are the kind of thing that is hard to see later:

• The report is built in the renderer, not over IPC. The first version rebuilt it on every keystroke, and each round trip did a synchronous file read on the process that draws the window. Making site-environment.cjs perform no I/O let the renderer import the formatters directly, which also means the preview and the clipboard come from one call with one set of inputs. The card says "see what will be copied", and two build paths would eventually make that a lie.
• The environment reads asynchronously. fs.readFileSync inside an ipcMain.handle freezes the window on a slow volume or behind an antivirus filter driver.

Three bugs found on the way

Reported separately, all pre-existing and none introduced by this work. I checked that rather than assuming it: the SQLite one reproduces on clean trunk with my changes removed, the CSP one is in lines my diff does not touch, and the tar one is in a file I have not modified at all.

1. #372: npm run build deletes the SQLite integration and the site stops booting. Reproduced on two sites.
2. #374: the renderer's two global error handlers never install, because the CSP blocks the inline scripts that add them.
3. #373: the build fails on Windows when Git for Windows is on PATH, because GNU tar reads C:\... as a remote host.

What I would like to know

1. Is this the right shape, or is the report better as something the app never touches?
2. Should the history live in electron-store alongside the site registry, or somewhere a contributor can see and edit?
3. Is "keep the report" worth the storage, or is copying it once enough?
4. Item 11 is a data-loss fix rather than a feature. If the rest of this is too much to take on, that one is worth having on its own.

Happy to send this as one pull request or eleven smaller ones, whichever is easier to review. Say which and I will shape it that way.

---

Built and tested on Windows 11 against trunk. The screenshots are from my own machine, so they carry my own site paths and ticket numbers rather than a scrubbed profile.

Choose one of the independent proposals first; the issue does not name a specific implementation file or test entry point. Read the relevant existing code under src/, including siteMeta and the terminal or site-screen entry points, and compare it with the described prototype. Done means the selected proposal works without taking on the other ten, with its visible behavior matching the stated screenshots and report workflow.