hasura / hasura/graphql-engine
Hasura having issue with connectivity handling
- Dominant language
- TypeScript
- Stars
- 32.1k
- Forks
- 3k
- PR merge metrics
- PR metrics pending
Description
### Version Information
Server Version: 2.0.10
### Environment
OSS
### What is the expected behaviour?
**Background description:**
Hasura deployed in Kubernetes cluster with horizontal autoscaling based on the CPU usage.
Dozens of microservices connecting to Hasura and querying for data on approx 800 requests per second with occasional bursts to 2k requests per second.
**Expected behaviour:**
Hasura being able to handle direct traffic / close redundant connections and perform gc without counting on client to do so.
### Keywords
HTTP timeout, CPU usage, resources usage, HTTP connection
### What is the current behaviour?
**Behaviour observed:**
1. Microservices connecting **DIRECTLY** to Hasura service, using HTTP and/or http2 cause massive spikes in CPU usage of the Hasura deployment, causing autoscaling to kick in and keep stable on maximum allowed. CPU usage does not go down and utilises maximum allowed. Postgresql server hasura connects to starts dropping connections due to 'too many clients'.
2. Microservices connecting to Hasura hidden behind traefik load balancer - no issues whatsoever, no CPU spikes, no issues with the number of connections to the Postgres either.
### Any possible solutions?
**Possible explanation:**
Hasura issues with handling HTTP connections, either with their timeout or closing them down, causing PostgreSQL connections pool to deplete. Whilst placed behind the load balancer ( traefik in my case ), the load balancer itself deals with the client connection handling appropriately and causes no issues.
Contributor guide
Assessment
This issue has not been assessed yet.