hasura / hasura/graphql-engine
Hasura down when a connected PG replica is unavailable, despite connected to main PG DB & another replica
- Dominant language
- TypeScript
- Stars
- 32.1k
- Forks
- 3k
- PR merge metrics
- PR metrics pending
Description
### Version Information
Server Version: hasuraci/graphql-engine-pro:v1.3.3-pro.3-read-replica-supported
### Environment
GCP cloudSQL (postgres instance + 2 replicas) connected to Hasura v1.3.3 instance in k8s cluster in the same GCP project
### What is the expected behaviour?
When a cloudsql PG DB replica instance undergoes maintenance and becomes unavailable, Hasura is expected to remain operational since it is still connected to (1) main cloudsql PG DB instance and (2) a second replica instance that is operational and not undergoing maintenance.
### Keywords
Hasura, Google Cloud Platform, GCP, cloudSQL, postgres, replica, maintenance, data connection
### What is the current behaviour?
Hasura v1.3.3 instance returned 503 error while any of its connected DB instances are undergoing maintenance and unavailable (last occurrence: 3mins downtime)
### How to reproduce the issue?
1. Connect Hasura v1.3.3 instance to 1 main PG cloudsql instance and 2 replica instances.
2. on one of the replicas: schedule maintenance or restart
3. query Hasura during replica DB maintenance or restart window.
### Any possible solutions?
Have Hasura remain operational given at least one operational DB connection, and send requests to available DB instances.
Contributor guide
Research direction
No files, tests, or entry points are named. Reproduce the failure with one primary and two replica PostgreSQL connections while restarting a replica, then trace connection health and request routing; done means Hasura remains available and uses the operational database connections.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, kubernetes, postgres
- Domain
- backend, cloud, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100