Azure / Azure/azure-relay-dotnet
High Request Volume Causes Timeout in Hybrid Connection Listener
- Dominant language
- C#
- Stars
- 41
- Forks
- 43
- PR merge metrics
- No merged PRs in 30d
Description
## Description
When the Hybrid Connection Listene rexperiences a high volume of requests in a short period (e.g., 1,000 requests within a minute), it encounters a timeout exception.
## Root Cause:
The issue seems to stem from how the` HybridHttpConnection` class in the Microsoft.Azure.Relay package handles incoming requests. Specifically, in the `CreateAsync `method, `Task.Run` is used to spawn a new task for each request.
During a period of request spikes, if the request handler takes a long (eg. 100ms) time to execute, this will quickly consume the readily available threads from the ThreadPool. A significant delay is thereafter introduced for the creation of each new thread, leading
to increasignly long response times.
Further adding to the problem is that Azure Relay itself has a maximum timeout of 60 seconds, causing the request to fail if a response has not been received in that time.
Normally a long running request handler in ASP.NET Core is created as an async method allowing I/O operations to not block the threads. However, `HybridConnectionClient.RequestHandler` is an `Action` preventing us from using async techniques to alleviate this problem.
Suggestion is to make `HybridConnectionListener.RequestHandler` async, and `await` it from within `ProcessFirstRequestAsync`. This would be a breaking change, so perhaps an alternative would be to introduce a new property `Func RequestHandlerAsync` that is used if it is set instead of the current `RequestHandler`
Contributor guide
Assessment
This issue has not been assessed yet.