IHostApplicationLifetime StopApplication not respected in AspNetCore.Mvc.Testing

# Describe the bug
I'm trying to run integration tests, with some things that call `IHostApplicationLifetime.StopApplication` from an IHostedService

In a "real project" this works, but it does not when using the TestServer.
Test case below

### To Reproduce
```csharp
namespace DefaultNamespace
{
using System.Threading;
using System.Threading.Tasks;
using BetterHostedServices.Test;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.AspNetCore.TestHost;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Xunit;

public class Test
{
// Dummy startup that does nothing
public class TestStartup
{
public void ConfigureServices(IServiceCollection services)
{
}

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
}
}

// Hosted Service that stops the application
public class HostedServiceThatShutsDown : IHostedService
{
private IHostApplicationLifetime lifetime;

public HostedServiceThatShutsDown(IHostApplicationLifetime lifetime)
{
this.lifetime = lifetime;
}

public Task StartAsync(CancellationToken cancellationToken)
{
this.lifetime.StopApplication();
return Task.CompletedTask;
}

public Task StopAsync(CancellationToken cancellationToken) => Task.CompletedTask;
}

// WebApplicationFactory with a few modifications so it can stand-alone
public class TestWebApplicationFactory : WebApplicationFactory where TStartup : class
{
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.UseSolutionRelativeContentRoot(
"./tests/IntegrationUtils"); // Just because I have a custom dir in my project, can disregard
}

//from https://stackoverflow.com/questions/61159115/asp-net-core-testing-no-method-public-static-ihostbuilder-createhostbuilders
protected override IHostBuilder CreateHostBuilder()
{
var builder = Host.CreateDefaultBuilder()
.ConfigureWebHostDefaults(x =>
{
x.UseStartup();
});
return builder;
}
}

[Fact]
public void IssueRepro()
{
var factory = new TestWebApplicationFactory()
.WithWebHostBuilder(b =>
b.ConfigureTestServices(services =>
{
services.AddHostedService();
}));

var client = factory.CreateClient();

// Works fine, but it shouldn't. The createClient above should error
client.GetAsync("/");
}
}
}
```

The IHostedService calls StopApplication. I then don't expect the `GetAsync` call to work then.
The logs look like this:

```
info: Microsoft.Hosting.Lifetime[0]
Application is shutting down...
info: Microsoft.Hosting.Lifetime[0]
Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
Hosting environment: Production
info: Microsoft.Hosting.Lifetime[0]
Content root path: /home/geewee/programming/BetterHostedServices/tests/IntegrationUtils
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://localhost/
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 18.5779ms 404
```

So it starts off by writing "Application is shutting down" and then proceeds to start the application.

### Further technical details
- ASP.NET Core version: 3.1
- Include the output of `dotnet --info`
```
.NET Core SDK (reflecting any global.json):
Version: 3.1.401
Commit: 39d17847db

Runtime Environment:
OS Name: fedora
OS Version: 32
OS Platform: Linux
RID: fedora.32-x64
Base Path: /usr/share/dotnet/sdk/3.1.401/

Host (useful for support):
Version: 3.1.7
Commit: fcfdef8d6b

.NET Core SDKs installed:
2.2.402 [/usr/share/dotnet/sdk]
3.0.103 [/usr/share/dotnet/sdk]
3.1.401 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.2.8 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.2.8 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.0.3 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.7 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.2.8 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 3.0.3 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
https://aka.ms/dotnet-download
```

- The IDE (VS / VS Code/ VS4Mac) you're running on, and it's version
Rider, but I don't think that matters. All of this is run from the Console.

Start by reproducing the test with WebApplicationFactory, TestServer, and HostedServiceThatShutsDown, then trace how IHostApplicationLifetime.StopApplication is handled during factory startup. Compare the observed startup and GetAsync behavior with the expected shutdown behavior; done means the test no longer serves the request after StopApplication is called.

Written by the indexing model from the issue text.