hasura / hasura/graphql-engine

Hasura metadata API slows down as amount of tables increases

Open
#9,989 4 comments 0 reactions 0 assignees View on GitHub
k/bug
Dominant language
TypeScript
Stars
32.1k
Forks
3k
PR merge metrics
PR metrics pending

Description

I have a multi-tenant Hasura application, and the tracking performance degrades with every increase in the number of the tables tracked in the database.

Is this intended behavior? How can I help my application perform tracking updates when there is a small schema change such as a new table in any given schema, so that it doesn't take 30 seconds to track the new table if I have 2000 other tables in the db?

### Version Information

Server Version: **v2.33.4**
CLI Version (for CLI related issue):

### Environment

Self hosted graphql container running in Docker
PostgreSQL 14.5 on x86_64-pc-linux-musl, compiled by gcc (Alpine 11.2.1_git20220219) 11.2.1 20220219, 64-bit

### What is the current behaviour?
Tracking tables/creating object/array relations takes an increasing amount of time as the amount of tables in the DB increases.

### What is the expected behaviour?
The time it takes to track a single table should not increase proportionally to the amount of tables contained in the database.

### How to reproduce the issue?

1. Fill up the database with more, tracked, tables; as the amount of tables increases, the time it takes to query the API increases as well.

Example using a python function that calls the Hasura metadata api using the `requests` library, and using the [pg_track_tables](https://hasura.io/docs/latest/api-reference/metadata-api/table-view/#metadata-pg-track-tables) tracks a single table after it has been created and added to the database:
```
# 0 tables in the DB
func: track_tables took: 0.53 sec
...
# +1700 tables in the DB (even in different schemas, amount of schemas doesn't matter)
func: track_tables took: 9.85 sec
```

### Keywords
Slow, track, tables, timeout

Contributor guide

Open the contributing guide

Research direction

Reproduce the slowdown through the Hasura metadata API using pg_track_tables and the provided Python requests example against PostgreSQL. Compare tracking time with zero tables and roughly 1,700 tracked tables in the Docker deployment. Done means identifying and addressing the scaling behavior so tracking one new table does not grow proportionally with the total table count.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, graphql, postgres, python
Domain
api, databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.