MSBuild Generally Inconsistent/Buggy Behavior

We are trying to leverage MSBuild 15 for our build system and have just hit bug after bug due to inconsistencies in the overall MSBuild implementation. It’s always been the case that it’s tricky dealing with differences between how devenv and command line MSBuilds work. In MSBuild 15 and with dotnet core MSBuild, it’s gotten so bad that we have ended up needing to write unit tests just to verify all the different build variations actually work. There are builds via

- devenv
- MSBuild.exe (VS Tools version)
- dotnet build
- dotnet pack (with implicit build)
- dotnet msbuild
- dotnet restore && dotnet build

Each one of the above behaves differently...in some cases, massively differently.

Then the above list gets multiplied by 2 because sln file builds behave totally different from (cs)proj file builds.

What I’d like to know is - which of the above build scenarios/models should we be designing for to get the best level of support and stability going forward? It’s borderline impossible to support them all.

Our basic desire is to have a build script that runs

dotnet pack my.sln -o artifacts

A test script that runs

dotnet test my.sln -a ...

And have Visual Studio builds work as well

Calling msbuild.exe would be equally okay vs. dotnet.exe if it actually worked.

It sounds easy but it’s so far from it....

With a simple solution consisting of:
- classic csproj
- sdk csproj targeting multiple frameworks (with GeneratePackageOnBuild set to true)
- a net461 console app that needs a PackageReference to the nuget package produced by the aforementioned csproj (must be a PackageReference because the package has custom msbuild targets)
- a pssproj with Pester tests
- a sqlproj

Basically, what we’ve found

- solution restores behave vastly differently between msbuild.exe /t:Restore, dotnet restore and Visual Studio restore. We can hack around it but calling nested MSBuilds in custom targets, but that will make one scenario work and break another... There are tons of issues filed around this behavior but it’s never been addressed. Here is one
https://github.com/NuGet/Home/issues/4627

Then there is the issue with calling pack on the sln when a csproj in it has GeneratePackageOnBuild set to true.

Then there is the issue that vstest via dotnet test doesn’t find PowerShell tests (only vstest.exe does)...but calling msbuild.exe /t:vstest doesn’t seem to be supported.

What’s a good strategy here?