A red visual check carries no evidence: test.yml shards upload no diff artifacts

test.yml's visual (N) shards hang on screenshot capture and fail whole shards with zero baseline mismatches. They have no retry, and they feed the required main check — so each occurrence blocks a merge.

PR #3090 adds the missing attempts: "3" to test-visual-label.yml, but not to these shards: they do not use .github/actions/run-visual-shard at all (direct vitest), so there is no retry input to switch on.

Evidence — three PRs, same signature, one of them cannot possibly be at fault

#3075 changes only packages/codemods. A codemod transform cannot alter rendered output, so an entire visual shard failing there is proof the cause is environmental rather than a diff. #3082's shard 3 went green on a re-run of the same commit, independently confirming it.

In every case the whole shard fails uniformly at roughly the 5-second mark and no pixel comparison completes — toMatchScreenshot never clears its stability gate, so it writes no reference/actual/diff. These jobs upload no visual-diffs-* artifacts either, so a reviewer sees a red required check with nothing to inspect and no way to tell a hang from a regression.

It is not the documented firefox hang

run-visual-shard's attempts input documents the flake as "Firefox intermittently hangs on a single file in headless CI". These jobs run webkit only — the #3075 and #3076 logs contain no firefox at all (--browser.name=webkit, 102 webkit matches, zero firefox). So the assumption that this job is structurally less exposed because it never starts firefox does not hold: it fails the same way on webkit, and not on a single file but across the entire shard.

What to decide

Retrofitting a retry here is not a one-line change like #3090 was — it means either restructuring these shards onto run-visual-shard or duplicating a retry loop, and it changes the failure semantics of the job the required main check aggregates. That deserves its own review, which is why it was deliberately excluded from #3090 rather than bundled in.

Worth weighing alongside it: the stability budget itself. toMatchScreenshot's 5000 ms is vitest's untouched default, while waitForPaintedContent right beside it deliberately uses 20 s with the comment that CI hardware is slower. A retry treats the symptom; the budget may be the cause.

Also consider making these jobs upload their diff artifacts. Their absence is what makes each occurrence cost a log read to classify.

Provenance

Observed across a batch of ten open PRs on 2026-09-02, alongside two other non-code CI failures in the same window: the preview deploy job failing on a 10 s ConnectTimeoutError to api.mittwald.de (five occurrences), and the e2e hang in #3089.

Start with the visual shard definitions in test.yml and compare them with .github/actions/run-visual-shard and test-visual-label.yml, which already expose the retry pattern. Read the screenshot path around toMatchScreenshot and waitForPaintedContent, then inspect the cited shard logs. Done means the chosen retry or stability-budget approach is reviewed for required-check semantics and failures provide visual-diffs artifacts for diagnosis.