dotnet / dotnet/aspnetcore

WebApplicationFactory does not support common output path.

Open
#54,243 1 comment 1 reaction 0 assignees View on GitHub
area-minimal area-mvc feature-mvc-testing
Dominant language
C#
Stars
38.4k
Forks
10.9k
Avg merge
2d 10h
Merged PRs (30d)
281

Description

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Describe the bug

When using the `UseCommonOutputDirectory` project property or the `OutputPath` MSBuild flag, `WebApplicationFactory` randomly fails to initialize, throwing an `InvalidOperationException`.

This happens because the `MvcTestingAppManifest.json` does not have a namespace, so the parallel project builds will override each others file, thereby creating a race condition that only succeeds when the correct manifest is written last.

### Expected Behavior

The tests should not randomly fail when using a common output path. There are numerous ways to achieve this, but here are a few suggestions (obviously not all of these may be viable):
- The `MvcTestingAppManifest.json` should be namespaced to avoid the issue entirely.
- The `WebApplicationFactoryContentRoot` attribute should be used if the manifest doesn't contain a valid path, not only when the manifest doesn't exist.
- The `WebApplicationFactory.SetContentRoot` method could be `protected virtual` to allow replacing the behavior. Alternatively, it should not be called before any of the configuration methods run, which would allow applying the `TEST_CONTENTROOT_*` setting before the exception is thrown.
- The `builder is null` branch of `WebApplicationFactory.EnsureServer` should have some overridable method to configure the deferred host before the content root is set.

### Steps To Reproduce

- Checkout the official [WebApplicationFactory example](https://github.com/dotnet/AspNetCore.Docs.Samples/tree/main/test/integration-tests/8.x/IntegrationTestsSample).
- Add a new project with some random test class and reference it from the test project.
- Set `UseCommonOutputDirectory` to `true` for all projects.
- Build the original project, then the new project. Observe how the `MvcTestingAppManifest.json` is now missing the entry for the original project.
- Run the tests without building. Observe how the tests throw an exception when the manifest is wrong.

### Exceptions (if any)

System.InvalidOperationException : Solution root could not be located using application root

### .NET Version

8.0.100

### Anything else?

NET SDK:
Version: 8.0.100
Commit: 57efcf1350
Workload version: 8.0.100-manifests.8d38d0cc

Laufzeitumgebung:
OS Name: Windows
OS Version: 10.0.22631
OS Platform: Windows
RID: win-x64
Base Path: C:\Program Files\dotnet\sdk\8.0.100\

Installierte .NET-Workloads:
Workload version: 8.0.100-manifests.8d38d0cc
[maui-windows]
Installationsquelle: VS 17.8.34330.188
Manifestversion: 8.0.3/8.0.100
Manifestpfad: C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.maui\8.0.3\WorkloadManifest.json
Installationstyp: Msi

[maccatalyst]
Installationsquelle: VS 17.8.34330.188
Manifestversion: 17.0.8478/8.0.100
Manifestpfad: C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.maccatalyst\17.0.8478\WorkloadManifest.json
Installationstyp: Msi

[ios]
Installationsquelle: VS 17.8.34330.188
Manifestversion: 17.0.8478/8.0.100
Manifestpfad: C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.ios\17.0.8478\WorkloadManifest.json
Installationstyp: Msi

[android]
Installationsquelle: VS 17.8.34330.188
Manifestversion: 34.0.43/8.0.100
Manifestpfad: C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.android\34.0.43\WorkloadManifest.json
Installationstyp: Msi

Host:
Version: 8.0.0
Architecture: x64
Commit: 5535e31a71

.NET SDKs installed:
8.0.100 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 6.0.25 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.25 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.27 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.25 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.27 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.14 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 8.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
x86 [C:\Program Files (x86)\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
Not set

global.json file:
Not found

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.