dotnet / dotnet/templating

`TemplateVerificationException.TemplateVerificationErrorCode` does not seem to be set correctly

Open
#8,593 0 comments 0 reactions 0 assignees View on GitHub
triage:transfer-to-sdk
Dominant language
No language data
Stars
1.7k
Forks
399
Avg merge
3d 5h
Merged PRs (30d)
39

Description

### Product

dotnet CLI (dotnet new)

### Describe The Bug

I was adding some tests for a template and fat-fingered some of the `TemplateVerifierOptions` in the process. I noticed the `TemplateVerificationException.TemplateVerificationErrorCode` does not match the error code reported in the exception message itself nor in the console output which is sent to the `ILogger`.

[The docs for the enum](https://github.com/dotnet/templating/blob/main/tools/Microsoft.TemplateEngine.Authoring.TemplateVerifier/TemplateVerificationErrorCode.cs#L7-L10) specify that these should correspond to the dotnet new [exit codes](https://github.com/dotnet/templating/blob/main/docs/Exit-Codes.md).

### To Reproduce

1. Create a folder containing a basic template - The template itself and its content is not important in this scenario. It just needs to exist and overall be valid (for one of the error cases)
2. Create a new xunit project to test the template
3. Add the `Microsoft.TemplateEngine.Authoring.TemplateVerifier` nuget package
4. Add a test for an "Invalid template path" scenario:
- Set `TemplateVerifierOptions.TemplatePath` to a non-existent path
- Assert that the `VerificationEngine.ExecuteAsync(options)` throws a `TemplateVerificationException`
- Assert that the exceptions `TemplateVerificationErrorCode` is `InstallFailed` (106)
5. Add a test for an "Invalid template name" scenario:
- Set the `TemplateVerifierOptions.TemplatePath` to point to the template folder from step 1
- Set the `TemplateVerifierOptions.TemplateName` to a non-existent name
- Assert that the `VerificationEngine.ExecuteAsync(options)` throws a `TemplateVerificationException`
- Assert that the exceptions `TemplateVerificationErrorCode` is `TemplateDoesNotExist` (103)
6. Run `dotnet test`:
- Observe that both tests fail because, in both cases, the `ErrorCode` is set to `InstantiationFailed`
- Observe that the exception `Message` include the error code `106` and `103` respectively.
- Observe that the console output reported to the `ILogger` also mention the `106` and `103` error codes.

Minimal reproduction repo: https://github.com/asser-dk/BugReport-DotnetTemplating-TemplateVerificationException

### dotnet Info

output
.NET SDK:
Version: 9.0.101
Commit: eedb237549
Workload version: 9.0.100-manifests.4a280210
MSBuild version: 17.12.12+1cce77968

Runtime Environment:
OS Name: Windows
OS Version: 10.0.22631
OS Platform: Windows
RID: win-x64
Base Path: C:\Program Files\dotnet\sdk\9.0.101\

.NET workloads installed:
[aspire]
Installation Source: VS 17.12.35527.113
Manifest Version: 8.2.2/8.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.aspire\8.2.2\WorkloadManifest.json
Install Type: Msi

[maui-windows]
Installation Source: VS 17.12.35527.113
Manifest Version: 9.0.0/9.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.maui\9.0.0\WorkloadManifest.json
Install Type: Msi

[maccatalyst]
Installation Source: VS 17.12.35527.113
Manifest Version: 18.1.9163/9.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.maccatalyst\18.1.9163\WorkloadManifest.json
Install Type: Msi

[ios]
Installation Source: VS 17.12.35527.113
Manifest Version: 18.1.9163/9.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.ios\18.1.9163\WorkloadManifest.json
Install Type: Msi

[android]
Installation Source: VS 17.12.35527.113
Manifest Version: 35.0.7/9.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.android\35.0.7\WorkloadManifest.json
Install Type: Msi

Configured to use loose manifests when installing new manifests.

Host:
Version: 9.0.0
Architecture: x64
Commit: 9d5a6a9aa4

.NET SDKs installed:
3.1.426 [C:\Program Files\dotnet\sdk]
7.0.410 [C:\Program Files\dotnet\sdk]
8.0.206 [C:\Program Files\dotnet\sdk]
8.0.307 [C:\Program Files\dotnet\sdk]
8.0.404 [C:\Program Files\dotnet\sdk]
9.0.101 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 9.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 9.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.20 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 8.0.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 8.0.11 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 9.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
x86 [C:\Program Files (x86)\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
Not set

global.json file:
Not found

Learn more:
https://aka.ms/dotnet/info

Download .NET:
https://aka.ms/dotnet/download

### Visual Studio Version

_No response_

### Additional context

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.