HarperFast / HarperFast/studio

e2e: @visual spec is dead coverage — excluded everywhere, no committed baseline

Open
#1,582 0 comments 0 reactions 0 assignees View on GitHub
e2e-failure
Dominant language
TypeScript
Stars
5
Forks
4
Avg merge
1d 8h
Merged PRs (30d)
40

Description

Noticed during the adversarial review of the e2e suite; not addressed in #1572.

`sign-in.anon.spec.ts` has a `@visual` screenshot assertion, but:

- `package.json`'s `test` script is `playwright test --grep-invert @visual`, and that is what **both** lanes invoke, so it never runs automatically.
- `test:visual` is not wired into any lane.
- No `*-linux.png` baseline is committed, so even a manual `pnpm test:visual` fails with "snapshot doesn't exist".

So it reads as implemented coverage (the README documents a baseline workflow) while exercising nothing. Either:

- generate and commit the Linux baseline (`npm run test:e2e:snapshots` at the repo root, which runs it in the canonical container), and decide where visual runs fit — probably not the 15-minute poll; or
- drop the spec and the README section until someone wants it.

Low priority, but worth resolving one way or the other so the suite's advertised coverage matches reality.

Contributor guide

Open the contributing guide

Research direction

Start with package.json, sign-in.anon.spec.ts, and the README section describing visual baselines. Run pnpm test:visual or the root npm run test:e2e:snapshots command in the canonical container, then inspect how both lanes invoke the test script. Done means the chosen retain-or-remove path is consistent: retained coverage has a committed Linux baseline and an explicit lane decision, while removed coverage is no longer advertised.

Written by the indexing model from the issue text.

Assessment

Tech stack
playwright, typescript
Domain
ci-cd, documentation, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.