dotnet / dotnet/sdk

Discussion: should we drop AwesomeAssertions (`.Should()`) in favor of MSTest-native assertions?

Open
#55,030 3 comments 0 reactions 0 assignees View on GitHub
Area-Tooling untriaged
Dominant language
C#
Stars
3.2k
Forks
1.3k
PR merge metrics
PR metrics pending

Description

## Summary

Now that the test suite is on **MSTest 4.x** (`MSTest.Sdk`, pinned `4.3.0-preview.*`) and we've already moved off FluentAssertions to its OSS fork **AwesomeAssertions** (`8.0.2`, still shipping as `FluentAssertions.dll`), it's worth a deliberate discussion about whether we want to keep a third-party fluent-assertion library at all, or standardize on MSTest's native `Assert` surface (which has grown substantially in v4).

This is a **discussion issue**, not a proposal to rip everything out tomorrow.

## Current state (data)

A scan of `test/**/*.cs` (excluding TestAssets/Snapshots/Approvals) found:

- **~11,800** `.Should().X(` call sites (AwesomeAssertions), the dominant assertion vocabulary in build/SDK integration tests.
- Heavy MSTest `Assert.*` usage in newer projects (TemplateEngine, dotnet-format, ApiCompatibility, HotReload, containers), where MSTest v4 assertions (`HasCount`, `IsEmpty`, `ContainsSingle`, `Contains`, `IsGreaterThan`, …) are already partly adopted.

Most-repeated `.Should()` shapes and their MSTest-native equivalents:

| AwesomeAssertions | Sites | MSTest v4 |
|---|---|---|
| `.Should().Be(x)` | ~2,507 | `Assert.AreEqual` |
| `.Should().Contain(x)` | ~1,031 | `Assert.Contains` / `StringAssert.Contains` |
| `.Should().BeTrue()/BeFalse()` | ~1,308 | `Assert.IsTrue/IsFalse` |
| `.Should().HaveCount(n)` | ~308 | `Assert.HasCount(n, …)` |
| `.Should().BeEmpty()` | ~301 | `Assert.IsEmpty` |
| `.Should().ContainSingle()` | ~110 | `Assert.ContainsSingle` |
| `.Should().NotBeNull()/BeNull()` | ~410 | `Assert.IsNotNull/IsNull` |

## What's genuinely repo-specific (the hard part)

The bulk of `.Should()` usage isn't generic — it's **command-result** and **filesystem** assertions backed by our own extension layer in `test/Microsoft.NET.TestFramework/Assertions` (`CommandResultAssertions`, `FileInfoAssertions`, `DirectoryInfoAssertions`, `StringAssertionsExtensions`):

- `result.Should().Pass()` — ~2,422
- `…HaveStdOutContaining(...)` — ~867
- `File/Dir.Should().Exist()` — ~480
- `result.Should().Fail()` — ~446
- `…NotHaveStdErr()` — ~372

These already have a **hand-written MSTest mirror** in one place (`test/TemplateEngine/.../CommandResultAssertions.MSTest.cs`), proving the direction is viable — but it's a one-off local copy, not shared.

We also have non-trivial custom matchers that have no MSTest equivalent and would need to be re-homed regardless of direction, e.g. `StringAssertionsExtensions.BeVisuallyEquivalentTo` (git-style diff output) and `BeVisuallyEquivalentToIfNotLocalized`.

## Points for discussion

1. **Do we want to drop the third-party fluent layer at all?** AwesomeAssertions is OSS and works; the main motivations to leave would be (a) one fewer external dependency to track/sign, (b) consistency with MSTest-native style already used in newer projects, (c) better failure messages from MSTest v4's specialized asserts.
2. **If yes, what's the migration unit?** The realistic enabler is to promote shared **MSTest-based** `CommandResultAssertions` + `FileInfo`/`DirectoryInfo` assertion helpers into `Microsoft.NET.TestFramework.MSTest`, so the ~4,000+ command/file assertions can migrate without each project rewriting them. Generic shapes can largely be handled mechanically.
3. **If no / not now,** should we at least settle a **style guideline** (e.g. "new tests use MSTest-native asserts; existing AwesomeAssertions stays") so we stop growing both vocabularies in parallel?
4. **Custom visual-diff matchers** (`BeVisuallyEquivalentTo`, etc.) — keep as MSTest helpers, or is there an MSTest-native pattern we'd prefer?

## Non-goals

- No big-bang rewrite. Any migration would be incremental and project-by-project.
- This is independent of the MSTEST0037 analyzer (already enabled via `MSTestAnalysisMode=Recommended` + `TreatWarningsAsErrors`), which governs MSTest `Assert.*` usage only.

cc @baronfel @JeremyKuhne @marcpopMSFT (output/UX considerations on failure messages)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the scan of test/**/*.cs and the assertion helpers under test/Microsoft.NET.TestFramework/Assertions, then compare the existing test/TemplateEngine/.../CommandResultAssertions.MSTest.cs mirror. Done means a recorded decision on retaining or migrating AwesomeAssertions, the migration unit or style guidance, and treatment of the visual-diff matchers.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
testing
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.