hasura / hasura/graphql-engine
Hasura metadata API slows down as amount of tables increases
- Dominant language
- TypeScript
- Stars
- 32.1k
- Forks
- 3k
- PR merge metrics
- PR metrics pending
Description
I have a multi-tenant Hasura application, and the tracking performance degrades with every increase in the number of the tables tracked in the database.
Is this intended behavior? How can I help my application perform tracking updates when there is a small schema change such as a new table in any given schema, so that it doesn't take 30 seconds to track the new table if I have 2000 other tables in the db?
### Version Information
Server Version: **v2.33.4**
CLI Version (for CLI related issue):
### Environment
Self hosted graphql container running in Docker
PostgreSQL 14.5 on x86_64-pc-linux-musl, compiled by gcc (Alpine 11.2.1_git20220219) 11.2.1 20220219, 64-bit
### What is the current behaviour?
Tracking tables/creating object/array relations takes an increasing amount of time as the amount of tables in the DB increases.
### What is the expected behaviour?
The time it takes to track a single table should not increase proportionally to the amount of tables contained in the database.
### How to reproduce the issue?
1. Fill up the database with more, tracked, tables; as the amount of tables increases, the time it takes to query the API increases as well.
Example using a python function that calls the Hasura metadata api using the `requests` library, and using the [pg_track_tables](https://hasura.io/docs/latest/api-reference/metadata-api/table-view/#metadata-pg-track-tables) tracks a single table after it has been created and added to the database:
```
# 0 tables in the DB
func: track_tables took: 0.53 sec
...
# +1700 tables in the DB (even in different schemas, amount of schemas doesn't matter)
func: track_tables took: 9.85 sec
```
### Keywords
Slow, track, tables, timeout
Contributor guide
Research direction
Reproduce the slowdown through the Hasura metadata API using pg_track_tables and the provided Python requests example against PostgreSQL. Compare tracking time with zero tables and roughly 1,700 tracked tables in the Docker deployment. Done means identifying and addressing the scaling behavior so tracking one new table does not grow proportionally with the total table count.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, graphql, postgres, python
- Domain
- api, databases, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100