Azure / Azure/azure-functions-host
ReadToEnd() throws exception on slow requests; introduce MinRequestBodyDataRate
- Dominant language
- C#
- Stars
- 2k
- Forks
- 482
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 36
Description
Some http requests from our client are sent to the Azure Function very slowly. This causes the Azure Function to throw the following exception:
```
Microsoft.Azure.WebJobs.Host.FunctionInvocationException : Exception while executing function: ReceiveDataHttpTrigger ---> Microsoft.AspNetCore.Connections.ConnectionResetException : The client has disconnected ---> System.IO.EndOfStreamException : Attempted to read past the end of the stream.
```
It would be great if the Azure Function Host has the flexibility to specify the ```MinRequestBodyDataRate```. This was the solution for a WebApi deployment as you see below.
#### Investigative information
- Function App version (1.0 or 2.0): 2.0
- Function name(s) (as appropriate): ReceiveDataHttpTrigger
- Invocation ID: 7e8ac8eb-fbee-408d-bbb4-597d45b56355
- Region: North Central US
##### Exception
```
Microsoft.Azure.WebJobs.Host.FunctionInvocationException : Exception while executing function: ReceiveDataHttpTrigger ---> Microsoft.AspNetCore.Connections.ConnectionResetException : The client has disconnected ---> System.IO.EndOfStreamException : Attempted to read past the end of the stream.
End of inner exception
at Microsoft.AspNetCore.Server.IIS.Core.IO.AsyncIOOperation.GetResult(Int16 token)
at async Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.ReadBody()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
at System.IO.Pipelines.Pipe.GetReadAsyncResult()
at System.IO.Pipelines.Pipe.DefaultPipeReader.GetResult(Int16 token)
at async Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.ReadAsync(Memory`1 memory,CancellationToken cancellationToken)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.AspNetCore.Server.IIS.Core.HttpRequestStream.ReadAsyncInternal(Memory`1 buffer,CancellationToken cancellationToken)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.AspNetCore.Server.IIS.Core.HttpRequestStream.Read(Byte[] buffer,Int32 offset,Int32 count)
at Microsoft.AspNetCore.Server.IIS.Core.WrappingStream.Read(Byte[] buffer,Int32 offset,Int32 count)
at Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream.Read(Byte[] buffer,Int32 offset,Int32 count)
at System.IO.StreamReader.ReadBuffer()
at System.IO.StreamReader.ReadToEnd()
```
#### Repro steps
Unfortunately we are unable to reproduce the problem. The client that interfaces with our Azure Function sends random requests at a very slow speed. This causes the ```System.IO.StreamReader.ReadToEnd()``` to fail on the request body.
What we investigated is that the same code deployed as a WebApi throws an error on the same```System.IO.StreamReder.ReadToEnd()```. However it provides us with some more detail.
```
Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Reading the request body timed out due to data arriving too slowly. See MinRequestBodyDataRate.
at Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException.Throw(RequestRejectionReason reason)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1MessageBody.PumpAsync()
at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
at System.IO.Pipelines.Pipe.GetReadAsyncResult()
at System.IO.Pipelines.Pipe.DefaultPipeReader.GetResult(Int16 token)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.MessageBody.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream.ReadAsyncInternal(Memory`1 buffer, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream.Read(Byte[] buffer, Int32 offset, Int32 count) at System.IO.StreamReader.ReadBuffer()
at System.IO.StreamReader.ReadToEnd()
```
#### Expected behavior
As with the other requests, we would expect the req.Body to be read correctly even if the request is sent very slowly.
#### Actual behavior
System.IO.StreamReader.ReadToEnd() throws an exception when http request is performed to slow.
#### Known workarounds
When we publish the code as a WebApi we have the option to disable the request data rate with the MinRequestBodyDataRate parameter. Since we applied this parameter no occurrences of the
```
.UseKestrel(options =>
{
options.Limits.MinRequestBodyDataRate = null;
})
```
It would be great if the Azure Function Hosts has a similar setting that would allow us to disable, set a different value than the default 240 bytes/sec.
#### Related information
Provide any related information
* C#
Source
The source below is simplified to show you the call that we make to read the request body. We encapsulated it into some custom methods/classes, etc.
```csharp
[FunctionName("ReceiveDataHttpTrigger")]
public static async Task RunAsync([HttpTrigger(AuthorizationLevel.Function, "post", Route = "data")]HttpRequest req, ExecutionContext context, ILogger log)
{
log.LogDebug("Request received for path={path}", req.Path.ToString());
var body = new StreamReader(req.Body, System.Text.Encoding.UTF8).ReadToEnd();
return new OkResult();
}
```
Contributor guide
Research direction
Start with the Azure Functions host's HTTP startup path and the existing Kestrel UseKestrel options, then trace how ReceiveDataHttpTrigger reads req.Body. Compare the requested MinRequestBodyDataRate behavior with the shown slow-request exception; done means the host exposes a configurable rate, including disabling it, so slow request bodies are handled as expected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, csharp
- Domain
- api, backend, cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100