block / block/buzz

Flaky: profile-hover smoke spec derives its expected value from a mid-transition sample

Open
#6,237 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
32.7k
Forks
4.3k
Avg merge
1d 13h
Merged PRs (30d)
253

Description

## Summary

`message-feedback-snapshots.spec.ts:97 "profile hover uses the channel hover surface"` derives its *expected* value at runtime from a mid-transition sample, so it fails nondeterministically. It has failed on `main` and on PRs that touch no desktop code.

## Evidence: the expectation moves, the received value does not

`Desktop Smoke E2E (3)`, all 3 retries in one job:

```
retry 0 Expected: "rgba(0, 0, 0, 0.027)" Received: "rgba(0, 0, 0, 0.04)"
retry 1 Expected: "rgba(0, 0, 0, 0.03)" Received: "rgba(0, 0, 0, 0.04)"
retry 2 Expected: "rgba(0, 0, 0, 0.016)" Received: "rgba(0, 0, 0, 0.04)"
```

A regression moves the *received* side. Here the received value is stable at the settled `0.04` and the test's own oracle is what varies — three different points on one interpolation curve.

## Mechanism (readable from source)

`desktop/tests/e2e/message-feedback-snapshots.spec.ts:97-114`:

```ts
const channel = page.getByTestId("channel-random");
await channel.hover();
const channelHoverColor = await channel.evaluate(
(element) => getComputedStyle(element).backgroundColor, // sampled mid-transition
);
await profile.hover();
await expect(profile).toHaveCSS("background-color", channelHoverColor);

await waitForAnimations(page); // line 110 — AFTER both reads
```

The channel row's computed background is read immediately after `hover()`, while the CSS transition toward `0.04` is still interpolating, so it captures an arbitrary intermediate alpha. That value is then asserted against the profile card, which has settled. `toHaveCSS` retries the *profile* side, which never changes, so every attempt burns.

The helper that would fix it is already imported and used in this file — it is just on the wrong side of the sample.

## Base-branch occurrences (not PR-specific)

| run | head | job | result |
|---|---|---|---|
| 32074856518 | `f64899e5d` | Desktop Smoke E2E (3) | same spec, same signature (`0.016`/`0.027`/`0.016` vs `0.04`) |
| 32155633872 | `17d455a9b` (docs-only PR) | Desktop Smoke E2E (3) | same spec, all 3 retries |

## Suggested fix

Either await settling before sampling:

```ts
await channel.hover();
await waitForAnimations(page);
const channelHoverColor = await channel.evaluate(...);
```

or assert the literal design token instead of a runtime-sampled value. A test that derives its expected value from live animated state cannot distinguish a real drift from a sample time.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.