Create a different queue for retroactive analysis
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 179
- PR merge metrics
- No merged PRs in 30d
Description
## Background
Related to #102 , it's always felt awkward to have the same SQS queue for both regular operation and for retroactive analysis:
- New binaries added to the BinaryAlert bucket might not be scanned for hours if a large retro scan is underway
- A `live_test` will fail for the same reason if a retro scan is running
- A `purge_queue` to stop a runaway retro scan will also drop new object events from the queue
- The number of concurrent analyzers needed for a retro scan vs. normal operation could be vastly different
These problems could all be fixed if there were just a different SQS queue for retroactive analysis!
## Desired Change
Create a new SQS queue specifically for retro scans. It's not clear whether Lambda will allow 2 different SQS event source mappings, but it's worth a shot.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the existing SQS queue and the retroactive scan Lambda configuration, including the Terraform deployment settings. Check whether two SQS event source mappings are supported, then verify that retro scans use a separate queue without delaying live events or affecting queue purging. Done means the separate queue is deployable and both workflows operate independently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python, terraform
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100