block / block/buzz

Desktop E2E: 'failed initial relay dial retries automatically' is order-dependent (passes only as the first test in the file)

Open
#6,242 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
32.7k
Forks
4.3k
Avg merge
1d 13h
Merged PRs (30d)
253

Description

## Summary

`failed initial relay dial retries automatically` in `desktop/tests/e2e/relay-reconnect.spec.ts` fails whenever **any** test precedes it in the file. It currently passes only because it is the first test in the file.

This surfaced while reviewing #3320, which adds a test above it. #3320 is not the cause — the defect reproduces on clean `main` with that PR entirely absent.

## Reproduction (on `main` @ `5694e78de`, PR absent)

Unmodified file:

```
14 passed
```

Insert a trivial no-op test immediately above `failed initial relay dial retries automatically`:

```ts
test("trivial preceding test", async ({ page }) => {
await installMockBridge(page);
await page.goto("/");
await expect(page.getByTestId("channel-general")).toBeVisible();
});
```

Result:

```
✓ trivial preceding test
✘ failed initial relay dial retries automatically
Error: page.evaluate: Error: Relay state seam is not installed.
at relay-reconnect.spec.ts:207
```

Content of the preceding test is irrelevant — I reproduced with a trivial body, and with an unrelated existing test. Moving the added test *below* it restores green (15/15).

## Why it matters

The failure mode is `__BUZZ_E2E_GET_RELAY_CONNECTION_STATE__` not being installed when the poll first runs, i.e. the test races bridge installation. As the first test in the file it wins that race; anywhere else it loses.

This makes a load-bearing reconnect test silently order-dependent: any PR that adds a test above it inherits a confusing red that looks like it was caused by that PR. It also means the test's guarantee is weaker than it appears.

## Suggested fix

Have the poll tolerate the seam not yet existing (return a sentinel rather than throwing) so `expect.poll` can retry, instead of throwing out of `page.evaluate` — the other call sites in the same file already use `__BUZZ_E2E_GET_RELAY_CONNECTION_STATE__?.()` for this reason. Line 207 is the only one that throws.

## Environment note (separate, minor)

`playwright.config.ts:186` uses `reuseExistingServer: !process.env.CI` against a fixed port `4173`. Two agents/worktrees running desktop E2E on one machine serve each other's `dist`, producing unrelated cross-worktree failures. Worth a per-worktree port or at least a doc note.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.