Evaluate MTP GitHub test annotations and job summaries

Problem

A failed test should be understandable from the GitHub Actions run without searching a long console log or downloading a report first. Evaluate Microsoft.Testing.Platform (MTP) reporting for useful failure annotations and job summaries.

This is a reporting improvement, not a test-framework migration.

Existing implementation

At the reviewed revision, unit tests produce Spekt JUnit reports and Coverlet coverage. The unit-test workflow uploads results and coverage to Codecov.

The Microsoft MTP reporting article describes GitHub annotations, grouped logs, and summaries. Its publication-time guidance identifies the GitHub reporter as preview and calls out compatibility with the MTP version supplied by the test adapter. Recheck current package status before implementation.

Proposed scope

Start with one representative NUnit test project and one GitHub Actions job.

• Determine the resolved MTP, NUnit adapter, and reporter versions and verify their compatibility.
• Add the GitHub reporter to the pilot through the repository's central package configuration.
• Enable useful failure annotations and a concise run summary in CI.
• Retain existing JUnit output and coverage consumers.
• Exercise success, failure, skipped tests, and a test-host failure to understand what evidence survives.
• Evaluate log noise, reporting overhead, and duplicated output before broader rollout.

If the pilot helps, apply a shared reporting policy across relevant projects and matrix jobs. A decision to defer due to preview stability is a valid evaluation outcome.

Acceptance criteria

• The supported dependency combination and reporter release status are recorded.
• A deliberately failing pilot test produces an actionable annotation and summary identifying the project/test and failure.
• Source locations are correct when available, with understandable fallback output otherwise.
• Passing and skipped tests are summarized without excessive annotation noise.
• Multiple projects/frameworks do not overwrite or ambiguously combine results.
• Reports remain accessible on failure and the original test exit status is preserved.
• Existing JUnit/Codecov consumers and local test execution continue to work.
• Behavior is checked for fork PRs and jobs without privileged tokens.
• The pilot outcome and rollout or deferral decision are documented.

Open questions

• Which summary and annotation features justify enabling by default?
• Should preview reporter packages be accepted now or deferred until stable?
• Are current workflow conditions preventing useful results from being uploaded when tests fail?
• Which CI jobs need separate summaries rather than aggregation?

Azure DevOps history-based triage, replacing Spekt/Codecov, and changing test frameworks are outside the initial scope.

Reference

• Test reporting in Microsoft.Testing.Platform: from red build to root cause

Original issue

Moved from arturcic/GitVersion#918. GitHub does not support a direct transfer between these repository owners.

Triage — 16 September 2026

• Area: CI test reporting.
• Milestone: 7.0.x; optional improvement, non-blocking for the v7.0 release.
• Classification: build.
• Next step: verify resolved MTP/adapter/reporter compatibility, then pilot one NUnit project and one Actions job before repository-wide rollout.
• Release scope: adopt only if failure visibility improves without disrupting existing JUnit and coverage consumers; defer if preview stability is unsuitable.
• Tracking: direct sub-issue of GitTools/GitVersion#5137. No dependency on the .NET 11 migration or SonarCloud coverage work has been established.

Inspect build/build/Tasks/Test/UnitTest.cs and .github/workflows/_unit_tests.yml, then determine the resolved MTP, NUnit adapter, and reporter versions and their compatibility. Pilot the reporter with one NUnit project and Actions job, exercising passing, failing, skipped, and host-failure cases while retaining JUnit and Codecov behavior. Done means the evidence and rollout or deferral decision are documented.