microsoft / microsoft/microsoft-ui-reactor

[Flaky test] OverlayLifecycle_Expire_* races on an 80ms margin — and 2 of the file's 4 delays cannot be converted to WaitFor

Open
#995 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C#
Stars
646
Forks
54
Avg merge
1d 3h
Merged PRs (30d)
84

Description

### What happened?

`OverlayLifecycle_Expire_GoneAfterHold` and `OverlayLifecycle_Expire_DictDrained` in `tests/Reactor.AppTests.Host/SelfTest/Fixtures/ReconcileHighlightOverlayLifecycleTests.cs` are gated by a blind `Task.Delay` with only an **80 ms margin**, in a file whose sibling fixture was already fixed for exactly this once (#210).

This is **not** the #988 watchdog. Runs exhibiting it complete with the TAP trailer present and a green tail; #988's signature is a truncated trailer with later fixtures `Skipped`. Please apply the three-channel discriminator before filing further sightings against #988 — *"single fixture fails, otherwise-green job, passes on re-run"* is the signature of **any** nondeterministic assertion and discriminates nothing on its own.

### The arithmetic

`Show_NoRefresh_ExpiresAfterHoldDuration` (L137-155):

```csharp
var (canvas, targets, _, overlay) = await SetupAsync(H, holdMs: 100); // L141
overlay.Show(canvas, targets, Array.Empty());
H.Check("OverlayLifecycle_Expire_AliveImmediately", overlay.LiveSpriteCount == 1); // L145

await Task.Delay(180); // L148
H.Check("OverlayLifecycle_Expire_GoneAfterHold", overlay.LiveSpriteCount == 0); // L149
H.Check("OverlayLifecycle_Expire_DictDrained", overlay.ActiveTargetCount == 0); // L151
```

**100 ms hold, 180 ms wait — an 80 ms margin**, and it must cover a timer tick *plus* a dispatcher wave on a contended runner.

Compare the sibling in the same file, `Show_RefreshExtendsLifetime` (L101-133), `holdMs: 800` with delays 500/400/500 — margins of **300/400/300**. Its own comment at L105-109 records why:

> `Previous 500/350/250 had only 150–250ms margin and flaked under load.`

So the file already contains the empirical finding that **150-250 ms is not enough on this runner** — and the Expire fixture sits at 80 ms.

### Why it recurred: #210 fixed the symptom, not the mechanism

#210 — *"flake: OverlayLifecycle_Show_RefreshExtendsLifetime timing race"*, closed 2026-05-19 — fixed the **neighbouring fixture in this same file** by widening its margins. It did not convert the pattern, and it did not touch `Show_NoRefresh_ExpiresAfterHoldDuration`. The same flake class then surfaced on the fixture that was left alone, at the tightest margin in the file.

Widening again would repeat that mistake. `TESTING.md` L82 is unambiguous, and this file is a direct violation of it:

> Always wait on a concrete idle signal — **never `Task.Delay()`**

### The part that needs care: only 2 of the 4 delays are convertible

This is the reason to fix it deliberately rather than sweep it, and it is the main thing a future contributor needs from this issue. **`Harness.WaitFor` evaluates its predicate *before* the first `Render`**, so converting a gate whose condition is already true at t=0 makes the assertion vacuous — it stops testing anything.

| line | delay | check it gates | shape | `WaitFor` safe? |
|---|---|---|---|---|
| — | — | `Expire_AliveImmediately` (L145) | must hold **now** | n/a — no delay |
| **L148** | 180 | `Expire_GoneAfterHold` (L149), `Expire_DictDrained` (L151) | converges 1 → 0 | ✅ **yes** — false at t=0 |
| L114 | 500 | `Refresh_AliveBeforeRefresh` (L115) | must **still** be 1 | ❌ **vacuous** — true at t=0 |
| L120 | 400 | `Refresh_AliveAfterRefresh` (L124) | must **still** be 1 | ❌ **vacuous** — true at t=0 |
| **L128** | 500 | `Refresh_ExpiresAfterFinalWindow` (L129) | converges 1 → 0 | ✅ **yes** — false at t=0 |

The distinction is direction, not style. An **eventual** assertion (`count reaches 0`) converges monotonically, so polling can only help and the predicate is false on entry — `WaitFor` is strictly correct. A **survival** assertion (`count is still 1 after a window`) is true on entry, so `WaitFor` returns immediately and the assertion degrades to *"the sprite exists right now"*, which is what `Expire_AliveImmediately` already covers. It would pass forever, including against a regression that expires the sprite instantly.

For L114/L120 the delay is not gating the observation — **the delay *is* the stimulus**. Elapsed time is the thing under test, so there is no idle signal to wait on and no `WaitFor` rewrite that preserves meaning.

### Suggested fix

**Two different fixes, and conflating them is why this file looks like one problem.**

1. **L148 and L128 → `Harness.WaitFor`.** Straightforward and removes the two tightest races, including the reported one:
```csharp
H.Check("OverlayLifecycle_Expire_GoneAfterHold",
await Harness.WaitFor(() => overlay.LiveSpriteCount == 0, maxPasses: 60, perPassMs: 10));
```
Keep the assertion on `WaitFor`'s **return value** — discarding it and asserting separately reintroduces a race, and throwing on timeout truncates TAP.

2. **L114 and L120 need a deterministic clock, not a wait.** The seam already exists: `ReconcileHighlightOverlay.TestHoldDurationOverrideMs` (L51) is settable, so the expiry duration is injectable. Making the *timer source* injectable in the same way would let both survival assertions be driven by advancing a fake clock, removing the wall-clock dependency entirely. Until that exists, widening L114/L120 is a stopgap, not a fix — and should be labelled as one so the next person doesn't read it as resolved.

### Verification this needs before it is trusted

- After converting L148, confirm the check still **fails** if expiry is disabled — the `WaitFor` short-circuit is precisely the trap that makes a converted gate silently vacuous, and it is not visible in a passing run.
- Confirm `Expire_AliveImmediately` still fails if the sprite is never created, so the pair genuinely brackets the lifetime rather than both asserting the same instant.

### Reactor version / commit

`main` as of 2026-07-31, unchanged by #971 — that PR deliberately scoped the `Task.Delay` assert-gates out and converted only fixed-delay gates whose direction made `WaitFor` safe.

### Notes

Related: #210 (same file, sibling fixture, closed by widening margins), #988 (selftest watchdog — **not** this; the discriminator is the TAP trailer).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tests/Reactor.AppTests.Host/SelfTest/Fixtures/ReconcileHighlightOverlayLifecycleTests.cs, especially Show_NoRefresh_ExpiresAfterHoldDuration and Show_RefreshExtendsLifetime, then read TESTING.md at L82 and inspect Harness.WaitFor. Preserve the immediate and survival assertions while converting only the eventual expiry checks; verify the relevant checks fail when expiry or sprite creation is disabled and that TAP output remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.