microsoft / microsoft/microsoft-ui-reactor

tests/Reactor.Compile.Analyzer.Tests runs in no CI workflow — 11 tests that cannot fail

Open Beginner friendly
#1,038 3 comments 0 reactions 0 assignees View on GitHub
testing
Dominant language
C#
Stars
646
Forks
54
Avg merge
1d 3h
Merged PRs (30d)
84

Description

## Summary

`tests/Reactor.Compile.Analyzer.Tests` is in `Reactor.slnx` and compiles, but is **not referenced by any GitHub Actions workflow**. Its **11 tests have never gated anything** — they are assertions that cannot fail, because nothing executes them in a context that reports a result.

Found by auditing *by shape* rather than waiting for a symptom, after #1011 discovered the same condition in `tests/Reactor.DocPipeline.Tests`.

## Evidence

Measured against `main` (`1ed2644e`):

```
.github/workflows/*.yml 15 files
workflows naming Reactor.Compile.Analyzer.Tests 0
workflows naming Reactor.DocPipeline.Tests 0 (being fixed by #1011)
```

Both projects reference `xunit` + `Microsoft.NET.Test.Sdk`, so they are genuine test projects rather than harnesses or sample apps:

| project | .cs files with tests | test attributes | wired into CI |
|---|---|---|---|
| `Reactor.DocPipeline.Tests` | 16 | **121** | no — **#1011 fixes this** |
| `Reactor.Compile.Analyzer.Tests` | 2 | **11** | **no — unfixed** |

The 11 tests in this project:

```
CustomEventDelegateTypeAnalyzerTests.cs 4 test attributes
REACTOR0050Tests.cs 7 test attributes
```

## Why this matters more than an ordinary coverage gap

A test that is absent is visible as absent. **A test that exists, compiles, is reviewable and greppable, and is never executed looks identical to a working one** — including to anyone auditing the tree for coverage of the analyzers it targets.

There is a second-order effect worth anticipating, observed on #1011: **wiring the project in made that PR's CI go red**, because seven previously-unexecuted tests failed. The PR looked like it had introduced regressions; it was simply the first change to be measured. Expect the same here — any failures surfaced by wiring this project in are pre-existing, and should not be attributed to whoever turns it on.

## Suggested fix

Mirror what #1011 does for the doc-pipeline project — add a step to `ci.yml`:

```yaml
- name: Compile-analyzer tests
run: dotnet test tests/Reactor.Compile.Analyzer.Tests/Reactor.Compile.Analyzer.Tests.csproj -p:Platform=x64 --nologo
```

Land it **after** #1011, so the two do not conflict in the same region of `ci.yml`.

## The generalisable check

`dotnet build Reactor.slnx` compiling a test project is **not** evidence that it runs in CI. The discriminating check is one command per project:

```
grep -rl "" .github/workflows/ # empty => it has never gated anything
```

Worth running once over every test project in the tree, and worth adding as a gate so a newly added test project cannot land unwired. A test asserting that every `*.Tests.csproj` in the solution appears in at least one workflow file would close the class rather than these two instances.

## Notes

- Naive globbing on paths containing `test` over-collects badly here — 81 matches, most of them perf harnesses (`StressPerf.*`, `PerfBench.*`, `Allocation.*`, `DeferredMount.*`) and sample apps, which legitimately do not run in CI. The discriminating filter is a test-framework reference in the `.csproj` plus `[Fact]`/`[Theory]`/`[TestMethod]` attributes in the sources.
- `Reactor.AppTests.ThirdPartyControls` references no test framework and is correctly excluded.

Contributor guide

Open the contributing guide

Research direction

Start with .github/workflows/ci.yml and compare its existing test steps with tests/Reactor.Compile.Analyzer.Tests/Reactor.Compile.Analyzer.Tests.csproj. Add the project’s CI test step after the doc-pipeline step from #1011, then run the specified dotnet test command and confirm all 11 tests execute and their result is reported by GitHub Actions.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, github-actions
Domain
ci-cd, testing
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.