microsoft / microsoft/microsoft-ui-reactor
tests/Reactor.Compile.Analyzer.Tests runs in no CI workflow — 11 tests that cannot fail
- Dominant language
- C#
- Stars
- 646
- Forks
- 54
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 84
Description
## Summary
`tests/Reactor.Compile.Analyzer.Tests` is in `Reactor.slnx` and compiles, but is **not referenced by any GitHub Actions workflow**. Its **11 tests have never gated anything** — they are assertions that cannot fail, because nothing executes them in a context that reports a result.
Found by auditing *by shape* rather than waiting for a symptom, after #1011 discovered the same condition in `tests/Reactor.DocPipeline.Tests`.
## Evidence
Measured against `main` (`1ed2644e`):
```
.github/workflows/*.yml 15 files
workflows naming Reactor.Compile.Analyzer.Tests 0
workflows naming Reactor.DocPipeline.Tests 0 (being fixed by #1011)
```
Both projects reference `xunit` + `Microsoft.NET.Test.Sdk`, so they are genuine test projects rather than harnesses or sample apps:
| project | .cs files with tests | test attributes | wired into CI |
|---|---|---|---|
| `Reactor.DocPipeline.Tests` | 16 | **121** | no — **#1011 fixes this** |
| `Reactor.Compile.Analyzer.Tests` | 2 | **11** | **no — unfixed** |
The 11 tests in this project:
```
CustomEventDelegateTypeAnalyzerTests.cs 4 test attributes
REACTOR0050Tests.cs 7 test attributes
```
## Why this matters more than an ordinary coverage gap
A test that is absent is visible as absent. **A test that exists, compiles, is reviewable and greppable, and is never executed looks identical to a working one** — including to anyone auditing the tree for coverage of the analyzers it targets.
There is a second-order effect worth anticipating, observed on #1011: **wiring the project in made that PR's CI go red**, because seven previously-unexecuted tests failed. The PR looked like it had introduced regressions; it was simply the first change to be measured. Expect the same here — any failures surfaced by wiring this project in are pre-existing, and should not be attributed to whoever turns it on.
## Suggested fix
Mirror what #1011 does for the doc-pipeline project — add a step to `ci.yml`:
```yaml
- name: Compile-analyzer tests
run: dotnet test tests/Reactor.Compile.Analyzer.Tests/Reactor.Compile.Analyzer.Tests.csproj -p:Platform=x64 --nologo
```
Land it **after** #1011, so the two do not conflict in the same region of `ci.yml`.
## The generalisable check
`dotnet build Reactor.slnx` compiling a test project is **not** evidence that it runs in CI. The discriminating check is one command per project:
```
grep -rl "" .github/workflows/ # empty => it has never gated anything
```
Worth running once over every test project in the tree, and worth adding as a gate so a newly added test project cannot land unwired. A test asserting that every `*.Tests.csproj` in the solution appears in at least one workflow file would close the class rather than these two instances.
## Notes
- Naive globbing on paths containing `test` over-collects badly here — 81 matches, most of them perf harnesses (`StressPerf.*`, `PerfBench.*`, `Allocation.*`, `DeferredMount.*`) and sample apps, which legitimately do not run in CI. The discriminating filter is a test-framework reference in the `.csproj` plus `[Fact]`/`[Theory]`/`[TestMethod]` attributes in the sources.
- `Reactor.AppTests.ThirdPartyControls` references no test framework and is correctly excluded.
Contributor guide
Research direction
Start with .github/workflows/ci.yml and compare its existing test steps with tests/Reactor.Compile.Analyzer.Tests/Reactor.Compile.Analyzer.Tests.csproj. Add the project’s CI test step after the doc-pipeline step from #1011, then run the specified dotnet test command and confirm all 11 tests execute and their result is reported by GitHub Actions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, github-actions
- Domain
- ci-cd, testing
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100