dotnet / dotnet/msbuild

dotnet build slow when using GeneratePathProperty="true" and $(Pkg...) variables

Open
#7,559 3 comments 0 reactions 0 assignees View on GitHub
Area: Performance triaged
Dominant language
C#
Stars
5.5k
Forks
1.5k
Avg merge
1d 13h
Merged PRs (30d)
133

Description

### Issue Description

`dotnet restore` and `dotnet build` take a long time to build a solution. The solution only has one project and uses `GeneratePathProperty="true"` and a `$(PkgPackage_Name)` variable with wildcard paths.

I have a net6.0 console project that have some NuGet dependencies on external tools (`NUnit.ConsoleRunner` in this example). Our CI expects to find the console runner executable in a specific place, and I want to copy it using the .csproj project. This .csproj reproduces the issue. `dotnet restore` and `dotnet build` take about 4 min in my system with this example:

```xml


Exe
net6.0
enable
enable








```

### Steps to Reproduce

* Unzip this [sample project](https://github.com/dotnet/msbuild/files/8532638/NugetPerformanceIssuesCore.zip)
* Run `dotnet restore`
* Run `dotnet build`

Both commands take about 4 min on my Windows laptop. If I remove the `**` from the `` tag, or if I hardwire the path instead of using the variable `$(PkgNUnit_ConsoleRunner)` it takes a couple of seconds to run as expected.

### Data
These are the execution times when not using `**`:
```
~\source\repos\NugetPerformanceIssuesCore
λ dotnet restore
Determining projects to restore...
Restored C:\Users\MyUser\source\repos\NugetPerformanceIssuesCore\NugetPerformanceIssuesCore\NugetPerformanceIssuesCore.csproj (in 267 ms).
~\source\repos\NugetPerformanceIssuesCore
λ dotnet build
Microsoft (R) Build Engine version 17.1.1+a02f73656 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

Determining projects to restore...
All projects are up-to-date for restore.
NugetPerformanceIssuesCore -> C:\Users\MyUser\source\repos\NugetPerformanceIssuesCore\NugetPerformanceIssuesCore\bin\Debug\net6.0\NugetPerformanceIssuesCore.dll

Build succeeded.
0 Warning(s)
0 Error(s)

Time Elapsed 00:00:03.65
```

And these when `**` is used to copy recursivelly. Please note that the "Determining projects to restore..." step is not timed by the dotnet command, but in my tests it took almost the same as `dotnet build`.

```
~\source\repos\NugetPerformanceIssuesCore
λ dotnet restore
Determining projects to restore...
Restored C:\Users\MyUser\source\repos\NugetPerformanceIssuesCore\NugetPerformanceIssuesCore\NugetPerformanceIssuesCore.csproj (in 335 ms).
~\source\repos\NugetPerformanceIssuesCore
λ dotnet build
Microsoft (R) Build Engine version 17.1.1+a02f73656 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

Determining projects to restore...
All projects are up-to-date for restore.
NugetPerformanceIssuesCore -> C:\Users\MyUser\source\repos\NugetPerformanceIssuesCore\NugetPerformanceIssuesCore\bin\Debug\net6.0\NugetPerformanceIssuesCore.dll

Build succeeded.
0 Warning(s)
0 Error(s)

Time Elapsed 00:03:45.57
```

### Analysis

I ran both commands while monitoring with [Procmon](https://docs.microsoft.com/en-us/sysinternals/downloads/procmon) and `dotnet` seems to be traversing the whole filesystem, as if `$(PkgNUnit_ConsoleRunner)\**\*.*` was resolved as `\**\*.*`. This does not happen with `$(PkgNUnit_ConsoleRunner)\*.*` or with `C:\Users\MyUser\.nuget\packages\nunit.consolerunner\3.15.0\**\*.*`, but of course both solutions are not desirable.

### Versions & Configurations

```
OS: Windows
Architecture: x64
MSBuild: Microsoft (R) Build Engine version 17.1.1+a02f73656 for .NET (It can be reproduced with 17.0 too)
```

I have reproduced the issue with net6.0 and net472 projects (as long as you use ``).

```
λ dotnet --info
.NET SDK (reflecting any global.json):
Version: 6.0.202
Commit: f8a55617d2

Runtime Environment:
OS Name: Windows
OS Version: 10.0.19043
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\6.0.202\

Host (useful for support):
Version: 6.0.4
Commit: be98e88c76

.NET SDKs installed:
2.1.202 [C:\Program Files\dotnet\sdk]
2.1.401 [C:\Program Files\dotnet\sdk]
2.1.402 [C:\Program Files\dotnet\sdk]
2.1.403 [C:\Program Files\dotnet\sdk]
2.1.526 [C:\Program Files\dotnet\sdk]
2.1.602 [C:\Program Files\dotnet\sdk]
2.1.700 [C:\Program Files\dotnet\sdk]
2.2.105 [C:\Program Files\dotnet\sdk]
2.2.202 [C:\Program Files\dotnet\sdk]
2.2.300 [C:\Program Files\dotnet\sdk]
3.1.412 [C:\Program Files\dotnet\sdk]
5.0.100 [C:\Program Files\dotnet\sdk]
5.0.401 [C:\Program Files\dotnet\sdk]
5.0.404 [C:\Program Files\dotnet\sdk]
6.0.100 [C:\Program Files\dotnet\sdk]
6.0.202 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.All 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.24 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.2.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.24 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.2.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.22 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.3-servicing-26724-03 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.24 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.2.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.22 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.23 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.22 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.23 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.10 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.13 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.14 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET runtimes or SDKs:
https://aka.ms/dotnet-download
```
### Regression?

I do not know.

Contributor guide

No contributing guide indexed for this repository

Research direction

Unzip the linked NugetPerformanceIssuesCore sample and run dotnet restore and dotnet build with the recursive $(PkgNUnit_ConsoleRunner)\**\*.* entry in its .csproj. Compare it with the wildcard and hardcoded-path variants, then trace how GeneratePathProperty and the package variable are evaluated. Done means recursive copying still works without traversing the whole drive or taking minutes.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
build-system, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.