dotnet / dotnet/aspnetcore

Performance regression for larger responses in AspNetCore 6.0 with HttpSys and NewtonsoftJson

Open
#40,579 8 comments 2 reactions 1 assignee Claimed by @davidfowl View on GitHub
area-mvc Priority:2
Dominant language
C#
Stars
38.4k
Forks
10.9k
Avg merge
2d 10h
Merged PRs (30d)
281

Description

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Describe the bug

When migrating our application from netcoreapp3.1 to net6.0 I noticed a significant performance regression. Our application uses Mvc Core with Newtonsoft Json for serialization/deserialization in requests/responses and is run on HttpSys on Windows. After migration, the net6.0 version is >10 times slower for larger responses (~256KiB).

Performance degradation happens for responses larger than 30KiB. The larger the response, the bigger the degradation.

This reproduces when the app runs on a remote machine. It does not reproduce when running with HttpSys locally (i.e. request to localhost). It also does not reproduce when app runs on a remote machine on Kestrel or IIS in-process or IIS out-of-process.

The issue is somehow related to interaction between `FileBufferingWriterStream` used by `NewtonsoftJsonOutputFormatter` and HttpSys. The version used in netcoreapp3.1 calls `DrainBufferAsync` on `response.Body` which uses `Stream.CopyToAsync(Stream)`. The version used in net6.0 calls `DrainBufferAsync` on `response.BodyWriter` which uses `PipeWriter.CopyFromAsync(Stream)`. The former reads the buffer file in 128KiB chunks, whereas the latter in 4KiB chunks.

If I manually replace `NewtonsoftJsonOutputFormatter` in net6.0 with my own version that differs only in the call to `DrainBufferAsync(response.Body)`, the issue is resolved.

### Expected Behavior

There should be no performance degradation.

### Steps To Reproduce

Create a basic web application:

```xml


netcoreapp3.1;net6.0




```

```csharp
// Program.cs
using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;

namespace WebApplication
{
public class Program
{
public static void Main(string[] args)
{
CreateHostBuilder(args).Build().Run();
}

public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseHttpSys();
webBuilder.UseStartup();
});
}

public class Startup
{
public Startup(IConfiguration configuration)
{
Configuration = configuration;
}

public IConfiguration Configuration { get; }

public void ConfigureServices(IServiceCollection services)
{
var options = services.AddControllers();
options.AddNewtonsoftJson();
}

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}

app.UseRouting();

app.UseAuthorization();

app.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
});
}
}

[ApiController]
[Route("[controller]")]
public class TestController : ControllerBase
{
[HttpGet]
public IEnumerable Get(int length = 128)
{
length *= 1024;
return Enumerable.Repeat(0, (length - 1) / 2 - 1)
.Concat(Enumerable.Repeat(length % 2 == 0 ? 10 : 0, 1));
}

[HttpGet("info")]
public object GetInfo()
{
return Environment.Version;
}
}
}
```

Build for netcoreapp3.1 and net6.0:
```
dotnet publish --configuration Release --self-contained --runtime win-x64 --framework net6.0
dotnet publish --configuration Release --self-contained --runtime win-x64 --framework netcoreapp3.1
```

Publish to remote machine and query it, e.g. `http://[host]/test?length=256` for 256KiB response.

I'm getting the following results for a Windows Azure VM (size D2s v3) in West Europe when querying from Europe:
- netcoreapp 3.1, 256KiB response - ~200ms
- net6.0, 256KiB response - ~2400ms

### Exceptions (if any)

_No response_

### .NET Version

6.0.2

### Anything else?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.