Flaky: profile-hover smoke spec derives its expected value from a mid-transition sample
- Dominant language
- Rust
- Stars
- 32.7k
- Forks
- 4.3k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 253
Description
## Summary
`message-feedback-snapshots.spec.ts:97 "profile hover uses the channel hover surface"` derives its *expected* value at runtime from a mid-transition sample, so it fails nondeterministically. It has failed on `main` and on PRs that touch no desktop code.
## Evidence: the expectation moves, the received value does not
`Desktop Smoke E2E (3)`, all 3 retries in one job:
```
retry 0 Expected: "rgba(0, 0, 0, 0.027)" Received: "rgba(0, 0, 0, 0.04)"
retry 1 Expected: "rgba(0, 0, 0, 0.03)" Received: "rgba(0, 0, 0, 0.04)"
retry 2 Expected: "rgba(0, 0, 0, 0.016)" Received: "rgba(0, 0, 0, 0.04)"
```
A regression moves the *received* side. Here the received value is stable at the settled `0.04` and the test's own oracle is what varies — three different points on one interpolation curve.
## Mechanism (readable from source)
`desktop/tests/e2e/message-feedback-snapshots.spec.ts:97-114`:
```ts
const channel = page.getByTestId("channel-random");
await channel.hover();
const channelHoverColor = await channel.evaluate(
(element) => getComputedStyle(element).backgroundColor, // sampled mid-transition
);
await profile.hover();
await expect(profile).toHaveCSS("background-color", channelHoverColor);
await waitForAnimations(page); // line 110 — AFTER both reads
```
The channel row's computed background is read immediately after `hover()`, while the CSS transition toward `0.04` is still interpolating, so it captures an arbitrary intermediate alpha. That value is then asserted against the profile card, which has settled. `toHaveCSS` retries the *profile* side, which never changes, so every attempt burns.
The helper that would fix it is already imported and used in this file — it is just on the wrong side of the sample.
## Base-branch occurrences (not PR-specific)
| run | head | job | result |
|---|---|---|---|
| 32074856518 | `f64899e5d` | Desktop Smoke E2E (3) | same spec, same signature (`0.016`/`0.027`/`0.016` vs `0.04`) |
| 32155633872 | `17d455a9b` (docs-only PR) | Desktop Smoke E2E (3) | same spec, all 3 retries |
## Suggested fix
Either await settling before sampling:
```ts
await channel.hover();
await waitForAnimations(page);
const channelHoverColor = await channel.evaluate(...);
```
or assert the literal design token instead of a runtime-sampled value. A test that derives its expected value from live animated state cannot distinguish a real drift from a sample time.
Contributor guide
Assessment
This issue has not been assessed yet.