dotnet / dotnet/aspnetcore

Yarp app crash intermittently

Open
#64,600 2 comments 0 reactions 0 assignees View on GitHub
area-networking
Dominant language
C#
Stars
38.4k
Forks
10.9k
Avg merge
2d 10h
Merged PRs (30d)
281

Description

### Describe the bug
Yarp app crash intermittently

I am working with a customer who experiencing Yarp app crash intermittently. We found the following exception becomes an unhandled exception and crashed the process. Microsoft.AspNetCore.Server.IIS.Core.IISHttpServer EventId: 2 SpanId: ac5c45d9259c5a3e TraceId: 2d5c66b8dd754f58a8706bed7786a0c5 ParentId: 0000000000000000 RequestId: 400f83b4-000a-7700-b63f-84710c7967bb RequestPath: /bd257a90-2cf9-4321-9f23-ca815cb0049a/v1/ProductSearchRequest Connection ID "8574853734269420467", Request ID "400f83b4-000a-7700-b63f-84710c7967bb":
```
An unhandled exception was thrown by the application. Exception: System.Runtime.InteropServices.COMException (0x800704CD):
An operation was attempted on a nonexistent network connection. (0x800704CD)
at Yarp.ReverseProxy.Forwarder.HttpForwarder.ResetOrAbort(HttpContext context, Boolean isCancelled)
at Yarp.ReverseProxy.Forwarder.HttpForwarder.HandleResponseBodyErrorAsync(HttpContext context, StreamCopyHttpContent requestContent, StreamCopyResult responseBodyCopyResult, Exception responseBodyException, ActivityCancellationTokenSource requestCancellationSource)
at Yarp.ReverseProxy.Forwarder.HttpForwarder.SendAsync(HttpContext context, String destinationPrefix, HttpMessageInvoker httpClient, ForwarderRequestConfig requestConfig, HttpTransformer transformer, CancellationToken cancellationToken)
at Yarp.ReverseProxy.Forwarder.ForwarderMiddleware.Invoke(HttpContext context)
at Relewise.Hosting.Proxy.Server.Infrastructure.Filters.RouteConfiguredIpFilteringMiddleware.InvokeAsync(HttpContext context) in C:\action-runners\api-cd-ci\_work\api.relewise.com\api.relewise.com\src\Relewise.Hosting.Proxy.Server\Infrastructure\Filters\RouteConfiguredIpFilteringMiddleware.cs:line 17
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Relewise.Hosting.Proxy.Server.Infrastructure.Logging.ExceptionLoggingMiddleware.Invoke(HttpContext context) in C:\action-runners\api-cd-ci\_work\api.relewise.com\api.relewise.com\src\Relewise.Hosting.Proxy.Server\Infrastructure\Logging\ExceptionLoggerMiddleware.cs:line 24 at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContextOfT`1.ProcessRequestAsync()
```

Customer is using the latest Yarp. from the call stack above, that looks the "System.Runtime.InteropServices.COMException" should be handled by Yarp code. At least not let it becomes an unhandled exception that crashes the w3wp.exe process.

### To Reproduce

exception and call stack see above
### Further technical details

- Include the version of the packages you are using: latest Yarp
- The platform (Linux/macOS/Windows)
Windows Server 2022

Contributor guide

Open the contributing guide

Research direction

Start at Yarp.ReverseProxy.Forwarder.HttpForwarder.ResetOrAbort and the surrounding response-body error path shown in the stack trace. Use the Windows Server 2022 crash details and exception code as the starting case; done means the intermittent COMException no longer becomes an unhandled process-crashing exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
backend-api-design, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.