NASA-AMMOS / NASA-AMMOS/plandev
Aerie Gateway Upload Large Dataset Loop
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 128
- Forks
- 33
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 12
Description
Checked for duplicates
Yes - I've already checked
Is this a regression?
No - This is a new bug
Version
develop
Describe the bug
Gateway is getting stuck in a loop when the user uploads an dataset file to /uploadDataset that contains a large number of profiles in a single file. The reason is that we place a limit on the query payload that gateway makes to hasura. That file size is determined by the number of profiles and the number of segments in each profile. The loop occurs when that limit is immediately reached by just instantiating the payload with all of the profiles even without the segments.
Reproduction
To repro, attempt to upload a dataset with at least 1000 profiles present.
Logs
System Info
Chrome
Severity
Moderate
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the gateway's /uploadDataset entry point and reproduce the issue with a dataset containing at least 1,000 profiles. Trace how the Hasura query payload limit is handled when the profile data alone reaches that limit. Done means a large dataset upload completes without looping and remains within the query payload limit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- graphql, java
- Domain
- api, backend, database
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100