Investigate possible build perf improvements in Copy task
- Dominant language
- C#
- Stars
- 5.5k
- Forks
- 1.5k
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 141
Description
### Context
Reported by @davkean:
> In 40% of builds, I see something that Eric was looking at; [synchronous calls to Copy taking out thread pool threads](https://prism.vsdata.io/failure/?query=tags%3Dttperfscenarios_solutionbuild%20bc!%3Dgc%20bc!%3Dhardfault%20threadtype%3D.NETThreadPool&eventType=blockedtime&failureType=dualdirection&failureHash=3cf89148-5518-7128-36c4-d660b78553b0). The CLR will take 500ms before it adds another thread to replace each one of these (so 5 seconds to replace 10 threads), so this is likely causing thread pool starvation and a non-optimal use of CPU usage while that occurs.
Due to the 'multi-process' nature of msbuild the threatpool starvation might not be such a big concern yet, but blocking the ability to run other waiting build work is of a big concern.
Additionaly there might be additional task in a same bucket - e.g. [`CopyRefAssembly`](https://github.com/dotnet/roslyn/blob/main/src/Compilers/Core/MSBuildTask/CopyRefAssembly.cs) from roslyn
### Suggested approach
- Attempt to perform couple experiments with implanting `TaskHost.Yield` into the Copy task (similarly to what Zip/Unzip does today) and evaluate the impact on the build
- Attempt to see if we can improve the current paralelized copy algorithm (possbily in connection with above)
### Goal
`Copy` is notoriously among the top time consumers within the builds
being able to use the consummed time for additional build tasks might help improve the CPU utilization and throughput of the build
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.