dotnet / dotnet/msbuild

[Flaky Test] Microsoft.Build.UnitTests.XMakeAppTests.GetCommandLineQuotedExe

Open
#14,192 1 comment 0 reactions 0 assignees View on GitHub
flaky-test
Dominant language
C#
Stars
5.5k
Forks
1.5k
Avg merge
1d 8h
Merged PRs (30d)
141

Description

**Flaky-test key** (automated de-duplication — do not edit):
```text
flaky-test-id: Microsoft.Build.UnitTests.XMakeAppTests.GetCommandLineQuotedExe
```

## Evidence

This test has been flagged as flaky by the automated flaky-test detector.

- **Distinct sources:** 3 (rolling `main` builds only)
- **Rolling build IDs:** [1464065](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1464065), [1475010](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1475010), [1481906](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1481906)
- **Total failures:** 4
- **Distinct error signatures:** 4 (indicating non-deterministic failure modes)
- **Legs:** FullOnWindows, FullOnWindows Release
- **TFMs:** net472
- **Assembly:** `Microsoft.Build.CommandLine.UnitTests`
- **First seen:** 2026-06-15
- **Last seen:** 2026-06-25
- **Sample build:** [1464065](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1464065)

### Representative error

```
Shouldly.ShouldAssertException : output
should contain (case insensitive comparison)
"D:\a\_work\1\s\artifacts\bin\Microsoft.Build.CommandLine.UnitTests\Debug\net472\MSBuild.exe /v:diag C:\Windows\system32\notepad.exe"
but was actually
"MSBuild version 18.9.0-ci-26315-01+4f9fffae9 for .NET Framework ..."
```

## Location

- **Fully-qualified name:** `Microsoft.Build.UnitTests.XMakeAppTests.GetCommandLineQuotedExe`
- **Assembly:** `Microsoft.Build.CommandLine.UnitTests`
- **Source:** `src/MSBuild.UnitTests/XMake_Tests.cs` (line ~1294)
- **Project:** `src/MSBuild.UnitTests/Microsoft.Build.CommandLine.UnitTests.csproj`

## Next steps

This test will be quarantined via `[ActiveIssue("https://github.com/dotnet/msbuild/issues/")]` (from `Microsoft.DotNet.XUnitV3Extensions`, namespace `Xunit`) — not `[Fact(Skip=...)]` — until the underlying nondeterminism is fixed. The quarantine pipeline (definition 344, `azure-pipelines/quarantine.yml`) will continue running this test on Windows/Linux/macOS across main's rolling builds to gather failure signal over time.

> Generated by [Flaky Test Triage](https://github.com/dotnet/msbuild/actions/runs/28288806840) · 405.3 AIC · ⊞ 34.9K · [◷](https://github.com/search?q=repo%3Adotnet%2Fmsbuild+is%3Aissue+%22gh-aw-workflow-call-id%3A+dotnet%2Fmsbuild%2Fflaky-test-detector.agent%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with Microsoft.Build.UnitTests.XMakeAppTests.GetCommandLineQuotedExe in src/MSBuild.UnitTests/XMake_Tests.cs around line 1294, and compare its expected command-line output with the representative failure. Review how the test is quarantined through azure-pipelines/quarantine.yml, then investigate the nondeterminism and verify the test no longer fails across the Windows net472 rolling builds.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
build-system, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.