flosell / flosell/trailscraper
Supports streams of events instead of large JSON documents to make select/generate more efficient
- Dominant language
- Python
- Stars
- 838
- Forks
- 38
- Avg merge
- 2m
- Merged PRs (30d)
- 14
Description
At the moment, `trailscraper select` produces a full JSON document with the same format as it would appear in a normal CloudTrail JSON file. `trailscraper generate` reads this format.
This is very flexible as it composes well with other tools that might be used to deal with this kind of data. However, it takes a while longer (and possibly generates more overhead) since each step needs to completely finish their operation before handing it off the the next step.
If we'd output the selected events one by one as we found them, generate could already get to work on them.
Next step:
1. Prototype if this would actually make a significant improvement to decide if it's worth complicating the usage over this
Contributor guide
Assessment
This issue has not been assessed yet.