duo-labs / duo-labs/cloudmapper

Warn before Memory Exhaustion

Open
#619 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
JavaScript
Stars
6.3k
Forks
836
PR merge metrics
No merged PRs in 30d

Description

Running `public` or `report` on a medium sized account that resulted in a 6.3GB data.json results in the processes being Killed by the system. At least on Docker for Mac OS X the default memory limit is 2GB. From the comments it appears the entire data.json is loaded into memory.

If there is a way to stream the file through the process and extract relevant data structures rather than load the entire thing that would be nice. I think I saw a reference to that approach in the code notes, but didn't see any enhancement requests so adding it here.

Alternately you could do a quick check on file size and available memory and bail with a user friendly warning.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.