Test results in AzDO report unqualified method names after MSTest migration
- Dominant language
- C#
- Stars
- 3.2k
- Forks
- 1.3k
- PR merge metrics
- PR metrics pending
Description
# Test results in AzDO report unqualified method names after MSTest migration
## Summary
After the SDK repo's migration from xUnit to MSTest (MTP), test results published to Azure DevOps report only the bare method name (e.g., `GetMemory_DefaultCtor`) instead of the fully-qualified name (e.g., `Microsoft.DotNet.HotReload.UnitTests.ArrayBufferWriterTests_Byte.GetMemory_DefaultCtor`). This makes test results significantly harder to interpret and blocks reliable test history lookup.
## Problem
When viewing test results in AzDO for SDK PR validation builds, the `automatedTestName` field contains only the short method name without namespace or class context. For example:
| xUnit behavior | MSTest/MTP behavior |
| --- | --- |
| `Microsoft.DotNet.Cli.Utils.Tests.GivenAProjectDependencyCommandResolver.ItFindsToolsLocatedInTheNuGetPackageFolderAsTool` | `ItFindsToolsLocatedInTheNuGetPackageFolderAsTool` |
| `Microsoft.DotNet.HotReload.UnitTests.ArrayBufferWriterTests_Byte.GetMemory_DefaultCtor` | `GetMemory_DefaultCtor` |
## Impact
### 1. Difficult to identify test failures
When a test fails, the class name provides essential context for knowing the area the failure occurs in and locating the source. A failure reported as `ItCanBuildTemplates` could be in any of dozens of test classes. Engineers must dig into work item logs to find the actual source.
### 2. Ambiguous test names
Multiple test classes can (and do) have methods with the same name. For example, `CountEqualsZero_FixedAsync` appears in multiple NetAnalyzers test classes. In AzDO, these show up as duplicate entries with no way to distinguish which class they belong to without digging into the logs.
### 3. Breaks historical test duration lookup
We are investigating adopting Roslyn's time-based Helix scheduling approach (bin-packing test methods into work items targeting a time budget using historical execution data). This requires looking up per-test durations from prior builds. With unqualified names:
- Duplicate method names across classes make duration data unreliable
## Ideal Experience
Ideally, the AzDO Tests tab would present results in an expandable hierarchy:
```
▶ Microsoft.DotNet.HotReload.UnitTests.ArrayBufferWriterTests_Byte (21 tests, 2 failed)
✓ ArrayBufferWriter_Ctor
✓ Advance
✗ GetMemory_DefaultCtor
✗ GetMemory_ExceedMaximumBufferSize_WithSmallStartingSize
...
▶ Microsoft.DotNet.Cli.Utils.Tests.GivenAProjectDependencyCommandResolver (8 tests)
✓ ItFindsToolsLocatedInTheNuGetPackageFolderAsTool
...
```
Class names grouped hierarchically would provide immediate context about which area of the codebase is affected, while still allowing drill-down into individual method results. Even if a full tree view isn't supported by AzDO's UI, simply having `automatedTestName` contain the FQN would at least allow grouping/filtering by class name prefix.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.