microsoft / microsoft/microsoft-ui-reactor
Selftest: HostCtrlFactory_Mounted/_PropsApplied flake under the coverage collector — the one fixed-delay gate 5a6bddb2 left unconverted
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 646
- Forks
- 54
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 84
Description
`HostCtrlFactory_Mounted` and `HostCtrlFactory_PropsApplied` fail intermittently in the **Merged coverage** job while passing in the uninstrumented selftest legs of the same commit. The mechanism is a fixed-delay timing gate, and it is the single site in its file that `5a6bddb2` ("test(selftest): replace fixed-delay assertion gates with WaitFor polling") converted none of while converting its five siblings.
## Observed
`Coverage` run [30690653808](https://github.com/microsoft/microsoft-ui-reactor/actions/runs/30690653808) on `41712d5c` (PR #1010):
```
not ok HostCtrlFactory_Mounted - assertion failed
not ok HostCtrlFactory_PropsApplied - assertion failed
```
The coverage script aborts before the merge step on any test failure, so this takes the whole job red via the `data_ok` gate:
```
Ensure coverage data WARNING: No valid coverage numbers (measure outcome=failure)
Reflect coverage status DATA_OK: false -> throw 'Coverage produced no valid numbers ...'
```
## The gate
`tests/Reactor.AppTests.Host/SelfTest/Fixtures/HostingCoverageFixtures.cs:143-167`
```csharp
var container = new Border { Child = hostControl };
H.SetContent(container);
await Harness.Render(200); // :159 fixed delay
var text = FindInContainer(hostControl, tb => tb.Text?.StartsWith("WithProps:") == true);
H.Check("HostCtrlFactory_Mounted", text is not null); // :162
H.Check("HostCtrlFactory_PropsApplied", text?.Text == "WithProps:test-value"); // :163
```
The in-source comment at `:153-156` states the assumption outright — *"The Loaded event fires asynchronously after the visual tree processes the addition. Use extra delay to ensure the mount + render completes"* — with a fixed 200 ms as the only thing enforcing it.
**Both checks fail together, which is the signature of `text == null`.** If the props had been wrong, `_Mounted` would pass and only `_PropsApplied` would fail. Both going red means the `TextBlock` was not there yet — a missed window, not a behavioural error.
Under a coverage collector everything runs slower, which is why this is the leg that trips.
## In-band control — same commit, two environments, opposite answers
Same commit `41712d5c`, same byte-identical fixture file:
| leg | result |
|---|---|
| `Selftests` / `AOT Selftests` (uninstrumented) | `ok HostCtrlFactory_Mounted` / `ok HostCtrlFactory_PropsApplied` (verified by name in the TAP output, not inferred from the job's green tick) |
| `Merged coverage` (instrumented) | `not ok` on both |
Verified by check name in the raw TAP rather than by the job conclusion, since "did not run" and "passed" are indistinguishable at job level.
## Why this site specifically
`5a6bddb2` converted 27 of 46 fixed-delay gates and deliberately kept 19. Its own rationale for keeping them:
> WaitFor evaluates its predicate BEFORE its first Render, so for exact-count and "state is unchanged" assertions the fixed delay IS the negative-observation window and converting would make them vacuous.
`HostCtrlFactory_Mounted` is not in that category and is not in the commit's list of deliberately-kept sites. It is an **eventual** assertion — `text is not null` is false at t=0 and converges once `Loaded` → `OnLoaded` → `Mount` completes — so a `WaitFor` conversion is sound here rather than vacuity-inducing. That distinction is worth stating explicitly, because the blanket reading of the caveat in `AGENTS.md` ("the obvious remediation can itself be the vacuous one") would otherwise argue against converting it.
In the same file, `HostCtrlFunc_Updated` at `:120-123` is already the correct shape.
## Suggested fix
Convert to the sibling idiom, keeping a total budget larger than the current 200 ms so tolerance never decreases:
```csharp
H.Check("HostCtrlFactory_Mounted", await Harness.WaitFor(
() => FindInContainer(hostControl, tb => tb.Text?.StartsWith("WithProps:") == true) is not null,
maxPasses: 24, perPassMs: 10));
var text = FindInContainer(hostControl, tb => tb.Text?.StartsWith("WithProps:") == true);
H.Check("HostCtrlFactory_PropsApplied", text?.Text == "WithProps:test-value");
```
`_PropsApplied` stays a plain comparison — it is the assertion with actual content, and it must not be folded into the wait predicate or it becomes unfalsifiable.
Mutation check to pair with it: drop `Props = "test-value"` from the `ReactorHostControl` initializer. `_PropsApplied` must go red while `_Mounted` stays green — which also proves the two checks are not measuring the same thing.
## Not from PR #1010
Recorded because that PR is where it surfaced, and the non-causation is structural rather than a file-list argument:
- `tests/Reactor.AppTests.Host/Reactor.AppTests.Host.csproj` has **no** `ProjectReference` to `tests/Reactor.Tests` — the selftest host cannot observe changes in the headless unit-test project.
- The entire delta between the coverage-green head `a3f04ed2` and the coverage-red head `41712d5c` is 2 files / +11/-8, both in `tests/Reactor.Tests/`.
- `HostingCoverageFixtures.cs` is byte-identical across those two heads (`git diff` = 0 bytes); it was last modified by `5a6bddb2`, which is on `main`.
## Wider point
The coverage job is the fleet's slowest leg, so it is the one that converts a marginal fixed delay into a red build — and it fails as a *coverage data* error (`DATA_OK: false`), several steps removed from the timing assumption that actually broke. Anyone reading the check name alone gets "coverage produced no numbers", which points at the coverage tooling rather than at a 200 ms settle in a hosting fixture. Grepping the remaining `Harness.Render()` gates in the selftest tree for *eventual* assertions still on a fixed delay would find the rest of this class before they surface the same way.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in tests/Reactor.AppTests.Host/SelfTest/Fixtures/HostingCoverageFixtures.cs:143-167 and compare the HostCtrlFactory checks with the WaitFor-based HostCtrlFunc_Updated case at :120-123. Run the relevant selftest and merged coverage jobs, then verify the mounted check tolerates delayed loading while the props check still detects a missing test-value; the suggested mutation should keep Mounted green and make PropsApplied fail.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100