Issue with HTTP Thread Utilization and Request Backlog Under High Load

Hi,

I am encountering a issue in my proxygen based http server application where I am using Proxygen HTTP threads as worker threads and Folly IO threads as consumer threads. The configuration includes the default HTTP server options, with a thread count of 240 for Proxygen and 384 for the IO worker threads.

Under high load, I observe that some HTTP threads are handling more than one request simultaneously, reaching up to 8 concurrent requests, while other HTTP threads are not fully utilized. This uneven load distribution is resulting in increased response times, as requests are placed in a wait state.

Additionally, I’ve noticed a significant backlog of gzip-compressed requests, and the load balancer seems to be directing more load to these specific threads.

Could you please advise on how to configure the system to limit the number of concurrent requests handled by each HTTP thread to a maximum of 3? Moreover, I’d like to prevent the accumulation of a specific type of request in the backlog.

Server Configuration: 48 core, Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz
Any insights or suggestions would be greatly appreciated.

Thank you.