dotnet / dotnet/msbuild

[Flaky Test] Microsoft.Build.Tasks.UnitTests.RoslynCodeTaskFactory_Tests.RoslynCodeTaskFactory_ReuseCompilation

Open
#14,418 3 comments 0 reactions 0 assignees View on GitHub
flaky-test
Dominant language
C#
Stars
5.5k
Forks
1.5k
Avg merge
1d 8h
Merged PRs (30d)
141

Description

**Flaky-test key** (automated de-duplication — do not edit):
```text
flaky-test-id: Microsoft.Build.Tasks.UnitTests.RoslynCodeTaskFactory_Tests.RoslynCodeTaskFactory_ReuseCompilation
```

## Evidence

- **Distinct sources:** 3 (3 rolling main builds)
- **Affected PRs:** none
- **Rolling build IDs:** [1496519](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1496519), [1506834](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1506834), [1514373](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1514373)
- **Legs / TFMs:** FullOnWindows, FullOnWindows Release / net472
- **Assembly:** Microsoft.Build.Tasks.UnitTests
- **Fully-qualified test name:** Microsoft.Build.Tasks.UnitTests.RoslynCodeTaskFactory_Tests.RoslynCodeTaskFactory_ReuseCompilation
- **First seen:** 2026-07-07 | **Last seen:** 2026-07-17
- **Distinct error hashes:** 3 (indicates nondeterministic failure)
- **Sample error:**
```
Shouldly.ShouldAssertException : env.CreateTestProjectWithFiles(...) assertion failure on build messages
```
- **Sample build:** https://dev.azure.com/dnceng-public/public/_build/results?buildId=1496519

## Notes

This test will be quarantined via `[ActiveIssue("")]` (from `Microsoft.DotNet.XUnitV3Extensions`, namespace `Xunit`) on a follow-up automated run once this issue has a real number. The quarantine stamps `Category=failing`, which normal CI excludes, and the quarantine pipeline (def 344) re-runs it on all platforms to gather signal over time. The underlying nondeterminism should be investigated and fixed by the auto-fixer workflow or a human.

> [!WARNING]
>
> Firewall blocked 1 domain
>
> The following domain was blocked by the firewall during workflow execution:
>
> - `awmgmcpg`
>> To allow these domains, add them to the `network.allowed` list in your workflow frontmatter:
>
> ```yaml
> network:
> allowed:
> - defaults
> - "awmgmcpg"
> ```
>
> See [Network Configuration](https://github.github.com/gh-aw/reference/network/) for more information.
>
>

> Generated by [Flaky Test Triage](https://github.com/dotnet/msbuild/actions/runs/29579390833) · 108 AIC · ⌖ 7.61 AIC · ⊞ 13.5K · [◷](https://github.com/search?q=repo%3Adotnet%2Fmsbuild+is%3Aissue+%22gh-aw-workflow-call-id%3A+dotnet%2Fmsbuild%2Fflaky-test-detector.agent%22&type=issues)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with Microsoft.Build.Tasks.UnitTests.RoslynCodeTaskFactory_Tests.RoslynCodeTaskFactory_ReuseCompilation and review its env.CreateTestProjectWithFiles(...) build-message assertion. Compare the three rolling builds and their errors, then reproduce the net472 failure on FullOnWindows. Done means the nondeterministic failure is understood and the test passes reliably without the quarantine masking it.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
build-system, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.