Sudden spike in TCP Close Wait
- Dominant language
- C#
- Stars
- 9.6k
- Forks
- 933
- Avg merge
- 12d 18h
- Merged PRs (30d)
- 2
Description
### Describe the bug
We started seeing a spike in TCP Close Wait / Http Queue Length (+4k) on one of our Azure App Services for Windows running YARP. Scaling out to one more instance (2->3) fixed the issue temporarily, after scaling down we had another spike. This is the first occurence of this issue in the last ~4 years we've been running YARP. Underlying infrastructure or code hasn't been changed in the last 2 months.
### Further technical details
- Azure App Service on Windows (P2v3), VNET Integration / PaaS access over Private Endpoints. Some calls are made to a AFD load-balanced service (Eg. public endpoint which are subject to SNAT restrictions)
- YARP 2.1, default HttpClient settings
- The AFD load-balanced service is reached with a typed HttpClient using a handler lifetime of 90 seconds (Down from 2 minutes)
- NET8
- YARP forwards requests to ~20 clusters (Unique destinations, also Azure App Services) with traffic being about 150 requests/s in total
- Requests on a few specific endpoints may take over 5-10 seconds, but those are limited.
Could there be a HttpClient setting we can optizime to release / close HttpClients earlier?
Contributor guide
Assessment
This issue has not been assessed yet.