BadHttpRequestException even before reaching controller action
- Dominant language
- C#
- Stars
- 38.4k
- Forks
- 10.9k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 290
Description
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Is your feature request related to a problem? Please describe the problem.
Hello, I am trying to debug a long going issue in one of our web services.
The issue is especially when under load our instances start throwing the exception:
```
Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Unexpected end of request content.
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1ContentLengthMessageBody.ReadAsyncInternal(CancellationToken cancellationToken)
at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream.ReadAsyncInternal(Memory`1 destination, CancellationToken cancellationToken)
at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
at System.Text.Json.Serialization.ReadBufferState.ReadFromStreamAsync(Stream utf8Json, CancellationToken cancellationToken, Boolean fillBuffer)
at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsObjectAsync(Stream utf8Json, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonInputFormatter.ReadRequestBodyAsync(InputFormatterContext context, Encoding encoding)
at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonInputFormatter.ReadRequestBodyAsync(InputFormatterContext context, Encoding encoding)
at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext bindingContext)
```
Throwing the exception is actually fine, and we want it to be consistent with the timeout token we send in (it is currently set to 1 minutes from our istio load balancer) we have tracing and through that I've seen some requests take unusual amount of time (sometimes up to 10-20 minutes) even before reaching the controller (I assume it happens while model binding)
I see in the stack trace it actually calls methods with cancellationtoken parametes, but I wonder if there is a problem which makes the service don't respect the cancellation?
I've created a custom wrapper around `BodyModelBinder` and added below cancellation logic to it and this reduced the number of long-running requests and we return a timeout response to the client, but (due to the nature of tasks in .net) model binding still tries to finish in the background, keeps using resources and eventually fails.
We are running .net8 on k8s 1.30 and we have istio load balancer in front of this service.
I wanted to ask if there is a better/proper way to cancel the request even before reaching the controller?
Thanks in advance,
Custom cancellation code:
```c#
public async Task BindModelAsync(ModelBindingContext bindingContext)
{
var token = CancellationTokenSource.CreateLinkedTokenSource(bindingContext.HttpContext.RequestAborted);
using (bindingContext.HttpContext.IsWarmup() ? null : _metrics?.Measure.BucketTimer.Time(SerializationMetrics.DeserializeTimerOptions))
{
var task = _actualModelBinder.BindModelAsync(bindingContext);
var delay = Task.Delay(TimeSpan.FromSeconds(5), token.Token);
if (await Task.WhenAny(delay, task) == task)
{
// Task completed within timeout.
// Consider that the task may have faulted or been canceled.
// We re-await the task so that any exceptions/cancellation is rethrown.
token.Cancel();
await task;
}
else if (bindingContext.HttpContext.RequestAborted.IsCancellationRequested)
{
throw new OperationCanceledException("Request cancelled.");
}
else
{
throw new TimeoutException("Model binding timed out.");
}
}
}
```
### Describe the solution you'd like
Possibly a more suitable overload of modelbinding or open to any suggestions
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.