jobizzness / jobizzness/bench

The phone screenshot harness gets rebuilt from scratch every time, and has a trap in it

Open
#67 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1
Forks
0
Avg merge
1h 21m
Merged PRs (30d)
41

Description

## Evidence

Three agents have now built the same thing. \`phone-design\` wrote \`shot.mjs\` and \`drive.mjs\` in \`.scratch-mock/\`; when that worktree was closed, \`phone-stage\` rebuilt an equivalent one and said so in its report:

> the harness worktree named in the ticket, \`phone-design-751985d0\`, no longer exists, so I rebuilt an equivalent one at \`.scratch-mock/\`

I have since copied \`shot.mjs\` into a report directory by hand to stop losing it a third time. It is untracked scratch in a worktree, so `bench close` also refuses to close finished tabs while it exists.

## The trap, which cost me three renders today

A fixture without `` lays out at **980px in headless Chrome regardless of `Emulation.setDeviceMetricsOverride`**. The screenshot comes out the requested 390px wide, so it looks right — but the page never entered the mobile media query, and every phone-only rule silently does nothing.

Measured, on a probe page that printed its own state:

```
width=980
match720=false
--broken=#d9705f
crashed=rgb(223, 232, 226) <- should have been --broken
```

With the meta tag added and nothing else changed, the same fixture rendered the crash line in `#d9705f` correctly.

This produced a **false negative on a working feature**: I reported the #64 crash tint as not applying, twice, before finding the cause. An agent hitting this would reasonably conclude its own CSS was broken and go rewrite working code.

`shot.mjs`'s own comment already warns about the neighbouring trap — `--window-size` floors around 500px and crops rather than shrinking — which is exactly the kind of note that would have saved the time here.

## Acceptance criteria

- [ ] The harness lives somewhere committed and shared, not in a per-worktree scratch directory
- [ ] It refuses, or loudly warns, when the page it loaded did not enter the mobile breakpoint — the failure must not be silent
- [ ] Its comments name both traps: the `--window-size` floor and the missing viewport meta
- [ ] A fixture template or helper that cannot be written without the viewport meta
- [ ] `pnpm typecheck` and `pnpm test` clean against the #50 baseline of 4 known failures

## Out of scope

- Turning it into a visual-regression suite that runs in CI. This is a tool for looking at things on purpose.
- Driving a real daemon; the stub is the right shape.

## Verification

```
pnpm typecheck
pnpm test # 4 failures expected: dispatch-modal, model-costs, model-picker, settings-ui (#50)
```

Manual: render a fixture without the viewport meta and confirm the harness says so rather than producing a confidently wrong picture.

## Related

- #64, #57, #47 (the three tickets that each needed this harness)

Contributor guide

Open the contributing guide

Research direction

Start by locating the existing .scratch-mock/shot.mjs and drive.mjs copies and the fixture or probe used to measure the mobile breakpoint. Read the harness comments and run pnpm typecheck and pnpm test against the stated #50 baseline. Done means a committed shared harness warns about missing viewport behavior, documents both traps, and uses a fixture template or helper that includes the viewport meta.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, typescript
Domain
testing-qa, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.