hasura / hasura/graphql-engine

Memory rise on `hasura metadata apply`

Open
#10,778 4 comments 1 reaction 0 assignees View on GitHub
k/bug
Dominant language
TypeScript
Stars
32.1k
Forks
3k
PR merge metrics
PR metrics pending

Description

### Version Information

Server Version:
`2.48.0`
CLI Version (for CLI related issue):
`version=2.48.0`

### Environment

EE

### What is the current behaviour?

When applying metadata with permissions and allow-rules, we see a significant rise in memory for the Hasura instance.

The largest files/directories that are imported are:
- Our query_collections file, around 2.5MB of size:
```
2.5M Sep 24 17:36 query_collections.yaml
```

- Our tables consist of 138 yaml files with a total of 1368K (1.37MB)
```
➜ tables ls -la
total 1368
```

Yet importing this enlarges the memory footprint at times to 6.5GB coming from a baseline of around 2GB.
We can ofcourse foresee enough memory, though it seems rather unusual to require this much memory during this action.

Similar to this report:
https://github.com/hasura/graphql-engine/issues/10601#issuecomment-2491657246

### What is the expected behaviour?

An fairly normal memory footprint considering the size of the metadata files.

### How to reproduce the issue?

1. Have tables & permissions combined with an allow list of queries
2. run metadata apply
3. Memory rise

### Screenshots or Screencast

Base footprint:
Image

Footprint during import:
Image

### Please provide any traces or logs that could help here.

Standard logging, nothing unusual here.

### Any possible solutions/workarounds you're aware of?

None

### Keywords

Hasura
CLI
Memory

Contributor guide

Open the contributing guide

Research direction

Start by reproducing `hasura metadata apply` with the permissions, allow-rules, query_collections.yaml, and table metadata described in the issue, while monitoring the Hasura instance's memory. Investigate where the import retains or expands this metadata; done means the same reproduction no longer causes the reported multi-gigabyte memory rise.

Written by the indexing model from the issue text.

Assessment

Tech stack
cli
Domain
backend, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.