Comfy-Org / Comfy-Org/ComfyUI_frontend

Vue-node text anti-aliasing makes @screenshot specs nondeterministic, ejecting PRs from the merge queue

Open
#15,358 3 comments 1 reaction 1 assignee Claimed by @DrJKL View on GitHub
area:testing
Dominant language
TypeScript
Stars
2k
Forks
702
Avg merge
1d 7h
Merged PRs (30d)
490

Description

## What

`@screenshot` e2e assertions that frame a Vue node containing a text widget are exposed to a text-rasterization nondeterminism that is not specific to any one spec. It has already ejected unrelated PRs from the merge queue.

## Evidence

Traced while fixing one instance in https://github.com/Comfy-Org/ComfyUI_frontend/pull/15356 (`pan.spec.ts` → `@mobile Can pan with touch`):

- The failure is always `40 pixels (ratio 0.01 of all image pixels) are different`, and the actual/diff PNGs are **byte-identical across different CI runners**.
- Every pixel outside the `ckpt_name` widget's value text is identical between passing and failing runs, including the node's own transform and border. The diff is anti-aliasing *inside the glyphs* of a `truncate`d value — identical glyph positions (per-glyph centroid drift < 0.09px), roughly 5% less ink.
- Playwright already captures a stable screenshot and re-fails all three retries with the same 40 pixels, so no readiness signal can wait it out.

## Why this is not one test's problem

Nothing in that mechanism is specific to panning. The conditions are: a text widget whose value is `truncate`d, inside a filtered and `contain`-ed node subtree, captured by an exact-match screenshot. Those conditions hold in every `@screenshot` test that frames a Vue node with a text widget.

## Impact

Not just red PR checks — silent merge-queue ejection of already-approved changes. Three queue branches on 2026-08-17 on the same base `926d78c8bf`: `pr-14068` and `pr-15067` both failed with the identical `40 pixels` (neither can move a canvas), while `pr-15108` passed on the same base.

## Suggested direction

Decide this once, centrally, rather than per spec as each one starts flaking:

- a default `maxDiffPixels` for `@screenshot` assertions that frame node text, sized well below a real regression (a 1px pan error moves 3615 px; an observed genuine rendering change moved 3146), or
- masking the widget value text where the assertion is about geometry rather than text, or
- pinning whatever in the font/rasterization stack is varying, if it can be identified.

15356 bounds the single known instance; this issue is for the shared cause.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.