adoptium / adoptium/aqa-test-tools
Improve $graphLookup query
- Dominant language
- Jupyter Notebook
- Stars
- 33
- Forks
- 97
- Avg merge
- 7h 9m
- Merged PRs (30d)
- 5
Description
As TRSS stores more and more data, we noticed MongoError: $graphLookup reached maximum memory consumption
```
MongoError: $graphLookup reached maximum memory consumption
at MessageStream.messageHandler (/home/testci/openjdk-test-tools/TestResultSummaryService/node_modules/mongodb/lib/cmap/connection.js:268:20)
at MessageStream.emit (events.js:198:13)
at processIncomingData (/home/testci/openjdk-test-tools/TestResultSummaryService/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
```
According to the [doc](https://docs.huihoo.com/mongodb/3.4/reference/limits/index.html), `The $graphLookup stage must stay within the 100 megabyte memory limit`. We need to look into improve our $graphLookup query efficiency.
Contributor guide
Research direction
Start by locating the TRSS query that uses $graphLookup and review the MongoDB memory-limit documentation linked in the issue. Reproduce the failure with sufficiently large data, then evaluate query changes for lower memory use. Done means the query completes without the reported maximum-memory error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mongodb
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100