Vs.net Stable, Rider Stable, VS Code Stable (less) periodically cause ports to not get closed - Crash on restart

### Is there an existing issue for this?

- [x] I have searched the existing issues

### Describe the bug

On VS.net, and Rider especially but also VS Code periodically, one or more of my projects (micro services) in the Aspire host will break on an error spinning up Kestrel and attaching to a port. Occasionally the port that doesn't get released is the Aspire Dashboard itself.

There doesn't seem to be any rhyme nor reason to it. It isn't about stop then start right away or clicking restart, it just doesn't release the ports and you have to close the IDE entirely and then it will work again on reopen. Very occasionally that doesn't work and you have to restart.

This happens on 3 separate Aspire projects that share only a little in common, and happens on Mac (VS Code and Rider) and Windows 11 Stable (all 3)

Once while using VS code, I looked in in the task manager and none of the dotnet processes had been killed. I killed them and restarted and it worked again. In Vs.net I don't see similar but am wondering if it's named something else when debugging so I don't see it.

I generally just click the stop button in the IDE, I don't close the aspire dashboard or anything else.

### Expected Behavior

Should regularly start without having to close the IDE and reopen.

### Steps To Reproduce

1. Create a marginally complex Aspire Host with database, Local AI (ollama), and Redis.
2. Make sure you have 3 or 4 .net 9 projects running minimal apis that spin up and use these things.
3. Start the aspire dashboard for debugging.
4. Stop the session using the IDE Stop button.
5. Start a new session with the start button. (or hit restart)
6. About once out of every 5-10 launches it will error with port conflicts with breakpoints into the kestrel startup.

### Exceptions (if any)

Port in use

### .NET Version info

```
.NET SDK:
Version: 9.0.301
Commit: a596cd22e2
Workload version: 9.0.300-manifests.f856936c
MSBuild version: 17.14.5+edd3bbf37

Runtime Environment:
OS Name: Windows
OS Version: 10.0.26100
OS Platform: Windows
RID: win-x64
Base Path: C:\Program Files\dotnet\sdk\9.0.301\

.NET workloads installed:
[android]
Installation Source: SDK 9.0.300, VS 17.14.36203.30
Manifest Version: 35.0.61/9.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.android\35.0.61\WorkloadManifest.json
Install Type: Msi

[aspire]
Installation Source: SDK 9.0.300, VS 17.14.36203.30
Manifest Version: 8.2.2/8.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.aspire\8.2.2\WorkloadManifest.json
Install Type: Msi

[ios]
Installation Source: SDK 9.0.300, VS 17.14.36203.30
Manifest Version: 18.4.9291/9.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.ios\18.4.9291\WorkloadManifest.json
Install Type: Msi

[maccatalyst]
Installation Source: SDK 9.0.300, VS 17.14.36203.30
Manifest Version: 18.4.9291/9.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.maccatalyst\18.4.9291\WorkloadManifest.json
Install Type: Msi

[maui-windows]
Installation Source: SDK 9.0.300, VS 17.14.36203.30
Manifest Version: 9.0.51/9.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.maui\9.0.51\WorkloadManifest.json
Install Type: Msi

[wasm-tools]
Installation Source: SDK 9.0.300, VS 17.14.36203.30
Manifest Version: 9.0.6/9.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.workload.mono.toolchain.current\9.0.6\WorkloadManifest.json
Install Type: Msi

Configured to use loose manifests when installing new manifests.

Host:
Version: 9.0.6
Architecture: x64
Commit: 3875b54e7b

.NET SDKs installed:
8.0.411 [C:\Program Files\dotnet\sdk]
9.0.301 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 8.0.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 9.0.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 8.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 9.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 8.0.17 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 9.0.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
x86 [C:\Program Files (x86)\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
Not set

global.json file:
E:\Repos\Api\global.json
```

### Anything else?

Our projects have this in them to configure kestrel. They use the default ports assigned by launchsettings.json however.

```
public static void ConfigureKestrel(this WebApplicationBuilder builder) =>
builder
.WebHost.ConfigureKestrel(
(context, options) =>
{
options.AddServerHeader = false;

options.Configure();

if (!builder.Environment.IsDevelopment() && !builder.Environment.IsTesting())
{
options.ConfigureHttpsDefaults(https =>
{
var certificateSettings =
builder.Configuration.GetRequiredConfiguration();
if (certificateSettings.Https is null)
{
throw new InvalidOperationException(
"Certificate settings for HTTPS are not configured."
);
}

https.ServerCertificate = certificateSettings.Https.GetCertificate();
});
}

options.ConfigureEndpointDefaults(listenOptions =>
{
listenOptions.Protocols = Microsoft
.AspNetCore
.Server
.Kestrel
.Core
.HttpProtocols
.Http1AndHttp2AndHttp3;
});
}
)
.UseSetting(WebHostDefaults.PreventHostingStartupKey, "true")
.UseSetting(WebHostDefaults.DetailedErrorsKey, "true")
.UseSetting(WebHostDefaults.CaptureStartupErrorsKey, "true")
.UseKestrelHttpsConfiguration();
```