dotnet / dotnet/sdk

[NET9] dotnet test hangs during build

Open
#45,461 7 comments 0 reactions 0 assignees View on GitHub
Area-dotnet test Area-dotnet test (VSTest) untriaged
Dominant language
C#
Stars
3.2k
Forks
1.3k
PR merge metrics
PR metrics pending

Description

### Describe the bug
When calling `dotnet test` the build step hangs on random projects.

```
C:\Users\indysingh\Desktop>dotnet-stack.exe report -p 20288
Thread (0x1958):
CPU_TIME
System.Private.CoreLib.il!System.Threading.WaitHandle.WaitOneNoCheck(int32,bool,class System.Object,value class WaitHandleWaitSourceMap)
System.Private.CoreLib.il!System.Threading.WaitHandle.WaitOne()
Microsoft.Build.il!Microsoft.Build.Execution.BuildSubmission.Execute()
MSBuild.il!Microsoft.Build.CommandLine.MSBuildApp.ExecuteBuild(class Microsoft.Build.Execution.BuildManager,class Microsoft.Build.Execution.BuildRequestData)
MSBuild.il!Microsoft.Build.CommandLine.MSBuildApp.BuildProject(class System.String,class System.String[],class System.String,class System.Collections.Generic.Dictionary`2,class System.Collections.Generic.Dictionary`2,class Microsoft.Build.Framework.ILogger[],value class Microsoft.Build.Framework.LoggerVerbosity,class Microsoft.Build.CommandLine.DistributedLoggerRecord[],int32,bool,class System.IO.TextWriter,class System.IO.TextWriter,bool,class System.Collections.Generic.ISet`1,class System.Collections.Generic.ISet`1,class System.Collections.Generic.ISet`1,bool,class Microsoft.Build.Logging.ProfilerLogger,bool,bool,value class Microsoft.Build.Execution.ProjectIsolationMode,class Microsoft.Build.Graph.GraphBuildOptions,bool,bool,bool,class System.String[],class System.String,bool,class Microsoft.Build.Execution.BuildResult&,class System.String[])
MSBuild.il!Microsoft.Build.CommandLine.MSBuildApp.Execute(class System.String[])
MSBuild.il!Microsoft.Build.CommandLine.MSBuildApp.Main(class System.String[])
Microsoft.DotNet.Cli.Utils.il!Microsoft.DotNet.Cli.Utils.MSBuildForwardingAppWithoutLogging.ExecuteInProc(class System.String[])
dotnet.il!Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingApp.Execute()
dotnet.il!Microsoft.DotNet.Tools.RestoringCommand.Execute()
dotnet.il!Microsoft.DotNet.Tools.Test.TestCommand.ForwardToMsbuild(class System.CommandLine.ParseResult,class System.String[],class System.String)
dotnet.il!Microsoft.DotNet.Tools.Test.TestCommand.Run(class System.CommandLine.ParseResult)
System.CommandLine.il!System.CommandLine.Invocation.InvocationPipeline.Invoke(class System.CommandLine.ParseResult)
System.CommandLine.il!System.CommandLine.ParseResult.Invoke()
dotnet.il!Microsoft.DotNet.Cli.Program.ProcessArgs(class System.String[],value class System.TimeSpan,class Microsoft.DotNet.Cli.Telemetry.ITelemetry)
dotnet.il!Microsoft.DotNet.Cli.Program.Main(class System.String[])
```

### To Reproduce
I really wish I could give you a step by step reproduction.

It's incredibly intermittent the exact command run 10 times will either fail consistently or succeed consistently.

```
dotnet test --tl:off --nologo --diag diag.log
```

diag*.log does have things like this:-

```
TpTrace Warning: 0 : 12940, 1, 2024/12/13, 21:19:04.426, 136452277481, vstest.console.dll, TestPluginDiscoverer: Failed to get types from assembly 'Microsoft.Diagnostics.NETCore.Client, Version=0.2.10.41601, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. Error: System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types.
Microsoft.Bcl.AsyncInterfaces, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the file specified.
at System.Reflection.RuntimeModule.GetDefinedTypes()
at System.Reflection.RuntimeModule.GetTypes()
at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginDiscoverer.GetTestExtensionsFromAssembly[TPluginInfo,TExtension](Assembly assembly, Dictionary`2 pluginInfos, String filePath) in /_/src/Microsoft.TestPlatform.Common/ExtensionFramework/TestPluginDiscoverer.cs:line 159
System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the file specified.

TpTrace Information: 0 : 12940, 1, 2024/12/13, 21:19:04.443, 136452451394, vstest.console.dll, Failed to load assembly Microsoft.VisualStudio.TestPlatform.Fakes, Version=17.0.0.0, Culture=neutral. Reason:System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.TestPlatform.Fakes, Version=17.0.0.0, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
File name: 'Microsoft.VisualStudio.TestPlatform.Fakes, Version=17.0.0.0, Culture=neutral, PublicKeyToken=null'
at System.Reflection.RuntimeAssembly.InternalLoad(AssemblyName assemblyName, StackCrawlMark& stackMark, AssemblyLoadContext assemblyLoadContext, RuntimeAssembly requestingAssembly, Boolean throwOnFileNotFound)
at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
at Microsoft.VisualStudio.TestPlatform.Common.Utilities.FakesUtilities.LoadTestPlatformAssembly() in /_/src/Microsoft.TestPlatform.Common/Utilities/FakesUtilities.cs:line 273
```

But that is seemingly a red-herring as that happens on a good run too.

It's like something gets "stuck" and remains stuck until we do a `dotnet build` _beforehand_ and then do `dotnet test --no-build` that _always_ works.

### Things we tried
- Eliminate transitive dependencies (because it has caused issues for us in the past)
- Upgrading nuget packages to latest version (things like TestPlaform etc)
- Ruled out application specific hangs (we have now seen the hang in a relatively small app compared to a large app - small and large referring to the number of tests)

### Exceptions (if any)
None that we can see so far.

### Further technical details
```
.NET SDK:
Version: 9.0.100
Commit: 59db016f11
Workload version: 9.0.100-manifests.c6f19616
MSBuild version: 17.12.7+5b8665660

Runtime Environment:
OS Name: Windows
OS Version: 10.0.20348
OS Platform: Windows
RID: win-x64
Base Path: C:\Program Files\dotnet\sdk\9.0.100\

.NET workloads installed:
There are no installed workloads to display.
Configured to use loose manifests when installing new manifests.

Host:
Version: 9.0.0
Architecture: x64
Commit: 9d5a6a9aa4

.NET SDKs installed:
6.0.418 [C:\Program Files\dotnet\sdk]
9.0.100 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 6.0.26 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.31 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 9.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.26 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.31 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 9.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 6.0.26 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 8.0.11 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 9.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
x86 [C:\Program Files (x86)\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
Not set

global.json file:
Not found

Learn more:
https://aka.ms/dotnet/info

Download .NET:
https://aka.ms/dotnet/download
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.