hasura / hasura/graphql-engine

Hasura having issue with connectivity handling

Open
#7,882 4 comments 2 reactions 0 assignees View on GitHub
k/bug
Dominant language
TypeScript
Stars
32.1k
Forks
3k
PR merge metrics
PR metrics pending

Description

### Version Information

Server Version: 2.0.10

### Environment

OSS

### What is the expected behaviour?

**Background description:**
Hasura deployed in Kubernetes cluster with horizontal autoscaling based on the CPU usage.
Dozens of microservices connecting to Hasura and querying for data on approx 800 requests per second with occasional bursts to 2k requests per second.

**Expected behaviour:**

Hasura being able to handle direct traffic / close redundant connections and perform gc without counting on client to do so.

### Keywords

HTTP timeout, CPU usage, resources usage, HTTP connection

### What is the current behaviour?

**Behaviour observed:**

1. Microservices connecting **DIRECTLY** to Hasura service, using HTTP and/or http2 cause massive spikes in CPU usage of the Hasura deployment, causing autoscaling to kick in and keep stable on maximum allowed. CPU usage does not go down and utilises maximum allowed. Postgresql server hasura connects to starts dropping connections due to 'too many clients'.

2. Microservices connecting to Hasura hidden behind traefik load balancer - no issues whatsoever, no CPU spikes, no issues with the number of connections to the Postgres either.

### Any possible solutions?

**Possible explanation:**
Hasura issues with handling HTTP connections, either with their timeout or closing them down, causing PostgreSQL connections pool to deplete. Whilst placed behind the load balancer ( traefik in my case ), the load balancer itself deals with the client connection handling appropriately and causes no issues.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.