duo-labs / duo-labs/cloudmapper
Warn before Memory Exhaustion
- Dominant language
- JavaScript
- Stars
- 6.3k
- Forks
- 836
- PR merge metrics
- No merged PRs in 30d
Description
Running `public` or `report` on a medium sized account that resulted in a 6.3GB data.json results in the processes being Killed by the system. At least on Docker for Mac OS X the default memory limit is 2GB. From the comments it appears the entire data.json is loaded into memory.
If there is a way to stream the file through the process and extract relevant data structures rather than load the entire thing that would be nice. I think I saw a reference to that approach in the code notes, but didn't see any enhancement requests so adding it here.
Alternately you could do a quick check on file size and available memory and bail with a user friendly warning.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.