YARP hostname routing + MapStaticAssets cause first request slowness and high memory usage
- Dominant language
- C#
- Stars
- 38.4k
- Forks
- 10.9k
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 276
Description
### Describe the bug
When YARP is used with ~200 or more hostname routes, and the ASP.NET Core service also uses `MapStaticAssets()`, the first request is very slow to execute, uses lots of cpu, and the memory usage of the service rises very high. This does not happen if either `MapReverseProxy()` or `MapStaticAssets()` is excluded, or if only path routes are used in YARP.
Profiling shows the culprit might be `DfaMatcher` initialization in the framework. Something about my usage of YARP causes it to blow up. This is an issue when running in resource-limited container environments: with limited CPU and memory the first request will take a _long_ time and the very high memory usage can cause OOMKills.
My service uses YARP to proxy traffic to a couple hundred of dynamically configured services, each routed based on an unique hostname. It also exposes it's own MVC UI on one hostname, and that UI needs static files. I'm using `MapStaticAssets().RequireHost(...)` to achieve this, because I couldn't figure out how to make the older `UseStaticFiles()` only respond on the single hostname and not on any of the YARP proxied hostnames.
Is there something that I can alter in the YARP configuration to avoid this slowness?
### To Reproduce
Full reproducer project here: https://github.com/luryus/yarp-host-explosion
This `Program.cs`, combined with ~20-40 static files under `wwwroot/` reproduces this issue.
```csharp
using Yarp.ReverseProxy.Configuration;
var builder = WebApplication.CreateBuilder(args);
const int routeCount = 300;
var routes = new List(routeCount);
var cluster = new ClusterConfig
{
ClusterId = "cluster",
Destinations = new Dictionary
{
{ "server", new DestinationConfig { Address = "http://127.0.0.1" } }
}
};
for (var i = 1; i <= routeCount; i++)
{
var hostName = $"route{i:D5}.example.com";
var routeId = $"route_{i}";
routes.Add(new RouteConfig
{
Order = -1000, RouteId = routeId, ClusterId = cluster.ClusterId,
Match = new RouteMatch { Hosts = [hostName] },
});
}
builder.Services.AddReverseProxy().LoadFromMemory(routes, [cluster]);
builder.Services.AddControllersWithViews();
builder.Services.AddHealthChecks();
var app = builder.Build();
app.UseRouting();
app.MapReverseProxy();
app.MapStaticAssets().RequireHost("localhost");
app.Run();
```
After the service has started up, make a request to `http://localhost:/css/1.css`. The first request takes over 5 seconds when running on my laptop, and the service memory usage rises to over 200 MB.
### Further technical details
- Linux & Windows
- YARP 2.3.0
- ASP.NET Core 9.0.10
Contributor guide
Assessment
This issue has not been assessed yet.