microsoft / microsoft/microsoft-ui-reactor

Selftest: HostCtrlFactory_Mounted/_PropsApplied flake under the coverage collector — the one fixed-delay gate 5a6bddb2 left unconverted

Open Beginner friendly
#1,041 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C#
Stars
646
Forks
54
Avg merge
1d 3h
Merged PRs (30d)
84

Description

`HostCtrlFactory_Mounted` and `HostCtrlFactory_PropsApplied` fail intermittently in the **Merged coverage** job while passing in the uninstrumented selftest legs of the same commit. The mechanism is a fixed-delay timing gate, and it is the single site in its file that `5a6bddb2` ("test(selftest): replace fixed-delay assertion gates with WaitFor polling") converted none of while converting its five siblings.

## Observed

`Coverage` run [30690653808](https://github.com/microsoft/microsoft-ui-reactor/actions/runs/30690653808) on `41712d5c` (PR #1010):

```
not ok HostCtrlFactory_Mounted - assertion failed
not ok HostCtrlFactory_PropsApplied - assertion failed
```

The coverage script aborts before the merge step on any test failure, so this takes the whole job red via the `data_ok` gate:

```
Ensure coverage data WARNING: No valid coverage numbers (measure outcome=failure)
Reflect coverage status DATA_OK: false -> throw 'Coverage produced no valid numbers ...'
```

## The gate

`tests/Reactor.AppTests.Host/SelfTest/Fixtures/HostingCoverageFixtures.cs:143-167`

```csharp
var container = new Border { Child = hostControl };
H.SetContent(container);
await Harness.Render(200); // :159 fixed delay

var text = FindInContainer(hostControl, tb => tb.Text?.StartsWith("WithProps:") == true);
H.Check("HostCtrlFactory_Mounted", text is not null); // :162
H.Check("HostCtrlFactory_PropsApplied", text?.Text == "WithProps:test-value"); // :163
```

The in-source comment at `:153-156` states the assumption outright — *"The Loaded event fires asynchronously after the visual tree processes the addition. Use extra delay to ensure the mount + render completes"* — with a fixed 200 ms as the only thing enforcing it.

**Both checks fail together, which is the signature of `text == null`.** If the props had been wrong, `_Mounted` would pass and only `_PropsApplied` would fail. Both going red means the `TextBlock` was not there yet — a missed window, not a behavioural error.

Under a coverage collector everything runs slower, which is why this is the leg that trips.

## In-band control — same commit, two environments, opposite answers

Same commit `41712d5c`, same byte-identical fixture file:

| leg | result |
|---|---|
| `Selftests` / `AOT Selftests` (uninstrumented) | `ok HostCtrlFactory_Mounted` / `ok HostCtrlFactory_PropsApplied` (verified by name in the TAP output, not inferred from the job's green tick) |
| `Merged coverage` (instrumented) | `not ok` on both |

Verified by check name in the raw TAP rather than by the job conclusion, since "did not run" and "passed" are indistinguishable at job level.

## Why this site specifically

`5a6bddb2` converted 27 of 46 fixed-delay gates and deliberately kept 19. Its own rationale for keeping them:

> WaitFor evaluates its predicate BEFORE its first Render, so for exact-count and "state is unchanged" assertions the fixed delay IS the negative-observation window and converting would make them vacuous.

`HostCtrlFactory_Mounted` is not in that category and is not in the commit's list of deliberately-kept sites. It is an **eventual** assertion — `text is not null` is false at t=0 and converges once `Loaded` → `OnLoaded` → `Mount` completes — so a `WaitFor` conversion is sound here rather than vacuity-inducing. That distinction is worth stating explicitly, because the blanket reading of the caveat in `AGENTS.md` ("the obvious remediation can itself be the vacuous one") would otherwise argue against converting it.

In the same file, `HostCtrlFunc_Updated` at `:120-123` is already the correct shape.

## Suggested fix

Convert to the sibling idiom, keeping a total budget larger than the current 200 ms so tolerance never decreases:

```csharp
H.Check("HostCtrlFactory_Mounted", await Harness.WaitFor(
() => FindInContainer(hostControl, tb => tb.Text?.StartsWith("WithProps:") == true) is not null,
maxPasses: 24, perPassMs: 10));

var text = FindInContainer(hostControl, tb => tb.Text?.StartsWith("WithProps:") == true);
H.Check("HostCtrlFactory_PropsApplied", text?.Text == "WithProps:test-value");
```

`_PropsApplied` stays a plain comparison — it is the assertion with actual content, and it must not be folded into the wait predicate or it becomes unfalsifiable.

Mutation check to pair with it: drop `Props = "test-value"` from the `ReactorHostControl` initializer. `_PropsApplied` must go red while `_Mounted` stays green — which also proves the two checks are not measuring the same thing.

## Not from PR #1010

Recorded because that PR is where it surfaced, and the non-causation is structural rather than a file-list argument:

- `tests/Reactor.AppTests.Host/Reactor.AppTests.Host.csproj` has **no** `ProjectReference` to `tests/Reactor.Tests` — the selftest host cannot observe changes in the headless unit-test project.
- The entire delta between the coverage-green head `a3f04ed2` and the coverage-red head `41712d5c` is 2 files / +11/-8, both in `tests/Reactor.Tests/`.
- `HostingCoverageFixtures.cs` is byte-identical across those two heads (`git diff` = 0 bytes); it was last modified by `5a6bddb2`, which is on `main`.

## Wider point

The coverage job is the fleet's slowest leg, so it is the one that converts a marginal fixed delay into a red build — and it fails as a *coverage data* error (`DATA_OK: false`), several steps removed from the timing assumption that actually broke. Anyone reading the check name alone gets "coverage produced no numbers", which points at the coverage tooling rather than at a 200 ms settle in a hosting fixture. Grepping the remaining `Harness.Render()` gates in the selftest tree for *eventual* assertions still on a fixed delay would find the rest of this class before they surface the same way.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in tests/Reactor.AppTests.Host/SelfTest/Fixtures/HostingCoverageFixtures.cs:143-167 and compare the HostCtrlFactory checks with the WaitFor-based HostCtrlFunc_Updated case at :120-123. Run the relevant selftest and merged coverage jobs, then verify the mounted check tolerates delayed loading while the props check still detects a missing test-value; the suggested mutation should keep Mounted green and make PropsApplied fail.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.