microsoft / microsoft/microsoft-ui-reactor
[Flaky test] OverlayLifecycle_Expire_* races on an 80ms margin — and 2 of the file's 4 delays cannot be converted to WaitFor
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 646
- Forks
- 54
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 84
Description
### What happened?
`OverlayLifecycle_Expire_GoneAfterHold` and `OverlayLifecycle_Expire_DictDrained` in `tests/Reactor.AppTests.Host/SelfTest/Fixtures/ReconcileHighlightOverlayLifecycleTests.cs` are gated by a blind `Task.Delay` with only an **80 ms margin**, in a file whose sibling fixture was already fixed for exactly this once (#210).
This is **not** the #988 watchdog. Runs exhibiting it complete with the TAP trailer present and a green tail; #988's signature is a truncated trailer with later fixtures `Skipped`. Please apply the three-channel discriminator before filing further sightings against #988 — *"single fixture fails, otherwise-green job, passes on re-run"* is the signature of **any** nondeterministic assertion and discriminates nothing on its own.
### The arithmetic
`Show_NoRefresh_ExpiresAfterHoldDuration` (L137-155):
```csharp
var (canvas, targets, _, overlay) = await SetupAsync(H, holdMs: 100); // L141
overlay.Show(canvas, targets, Array.Empty());
H.Check("OverlayLifecycle_Expire_AliveImmediately", overlay.LiveSpriteCount == 1); // L145
await Task.Delay(180); // L148
H.Check("OverlayLifecycle_Expire_GoneAfterHold", overlay.LiveSpriteCount == 0); // L149
H.Check("OverlayLifecycle_Expire_DictDrained", overlay.ActiveTargetCount == 0); // L151
```
**100 ms hold, 180 ms wait — an 80 ms margin**, and it must cover a timer tick *plus* a dispatcher wave on a contended runner.
Compare the sibling in the same file, `Show_RefreshExtendsLifetime` (L101-133), `holdMs: 800` with delays 500/400/500 — margins of **300/400/300**. Its own comment at L105-109 records why:
> `Previous 500/350/250 had only 150–250ms margin and flaked under load.`
So the file already contains the empirical finding that **150-250 ms is not enough on this runner** — and the Expire fixture sits at 80 ms.
### Why it recurred: #210 fixed the symptom, not the mechanism
#210 — *"flake: OverlayLifecycle_Show_RefreshExtendsLifetime timing race"*, closed 2026-05-19 — fixed the **neighbouring fixture in this same file** by widening its margins. It did not convert the pattern, and it did not touch `Show_NoRefresh_ExpiresAfterHoldDuration`. The same flake class then surfaced on the fixture that was left alone, at the tightest margin in the file.
Widening again would repeat that mistake. `TESTING.md` L82 is unambiguous, and this file is a direct violation of it:
> Always wait on a concrete idle signal — **never `Task.Delay()`**
### The part that needs care: only 2 of the 4 delays are convertible
This is the reason to fix it deliberately rather than sweep it, and it is the main thing a future contributor needs from this issue. **`Harness.WaitFor` evaluates its predicate *before* the first `Render`**, so converting a gate whose condition is already true at t=0 makes the assertion vacuous — it stops testing anything.
| line | delay | check it gates | shape | `WaitFor` safe? |
|---|---|---|---|---|
| — | — | `Expire_AliveImmediately` (L145) | must hold **now** | n/a — no delay |
| **L148** | 180 | `Expire_GoneAfterHold` (L149), `Expire_DictDrained` (L151) | converges 1 → 0 | ✅ **yes** — false at t=0 |
| L114 | 500 | `Refresh_AliveBeforeRefresh` (L115) | must **still** be 1 | ❌ **vacuous** — true at t=0 |
| L120 | 400 | `Refresh_AliveAfterRefresh` (L124) | must **still** be 1 | ❌ **vacuous** — true at t=0 |
| **L128** | 500 | `Refresh_ExpiresAfterFinalWindow` (L129) | converges 1 → 0 | ✅ **yes** — false at t=0 |
The distinction is direction, not style. An **eventual** assertion (`count reaches 0`) converges monotonically, so polling can only help and the predicate is false on entry — `WaitFor` is strictly correct. A **survival** assertion (`count is still 1 after a window`) is true on entry, so `WaitFor` returns immediately and the assertion degrades to *"the sprite exists right now"*, which is what `Expire_AliveImmediately` already covers. It would pass forever, including against a regression that expires the sprite instantly.
For L114/L120 the delay is not gating the observation — **the delay *is* the stimulus**. Elapsed time is the thing under test, so there is no idle signal to wait on and no `WaitFor` rewrite that preserves meaning.
### Suggested fix
**Two different fixes, and conflating them is why this file looks like one problem.**
1. **L148 and L128 → `Harness.WaitFor`.** Straightforward and removes the two tightest races, including the reported one:
```csharp
H.Check("OverlayLifecycle_Expire_GoneAfterHold",
await Harness.WaitFor(() => overlay.LiveSpriteCount == 0, maxPasses: 60, perPassMs: 10));
```
Keep the assertion on `WaitFor`'s **return value** — discarding it and asserting separately reintroduces a race, and throwing on timeout truncates TAP.
2. **L114 and L120 need a deterministic clock, not a wait.** The seam already exists: `ReconcileHighlightOverlay.TestHoldDurationOverrideMs` (L51) is settable, so the expiry duration is injectable. Making the *timer source* injectable in the same way would let both survival assertions be driven by advancing a fake clock, removing the wall-clock dependency entirely. Until that exists, widening L114/L120 is a stopgap, not a fix — and should be labelled as one so the next person doesn't read it as resolved.
### Verification this needs before it is trusted
- After converting L148, confirm the check still **fails** if expiry is disabled — the `WaitFor` short-circuit is precisely the trap that makes a converted gate silently vacuous, and it is not visible in a passing run.
- Confirm `Expire_AliveImmediately` still fails if the sprite is never created, so the pair genuinely brackets the lifetime rather than both asserting the same instant.
### Reactor version / commit
`main` as of 2026-07-31, unchanged by #971 — that PR deliberately scoped the `Task.Delay` assert-gates out and converted only fixed-delay gates whose direction made `WaitFor` safe.
### Notes
Related: #210 (same file, sibling fixture, closed by widening margins), #988 (selftest watchdog — **not** this; the discriminator is the TAP trailer).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tests/Reactor.AppTests.Host/SelfTest/Fixtures/ReconcileHighlightOverlayLifecycleTests.cs, especially Show_NoRefresh_ExpiresAfterHoldDuration and Show_RefreshExtendsLifetime, then read TESTING.md at L82 and inspect Harness.WaitFor. Preserve the immediate and survival assertions while converting only the eventual expiry checks; verify the relevant checks fail when expiry or sprite creation is disabled and that TAP output remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100