airbnb / airbnb/streamalert

Update kinesis_events module to allow sourcing events from multiple streams

Open
#771 2 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.9k
Forks
324
PR merge metrics
No merged PRs in 30d

Description

## Background

At the moment the `kinesis_events` terraform module configures a Lambda event source mapping only for the primary kinesis stream that the streamalert terraform playbook creates.

Ideally this module would instead configure a variable number of event source mappings depending on the streams configured in the `sources.json` config, such that the process of adding a new streamalert data source is simply:
1. update `sources.json` with the appropriate kinesis stream name
2. execute `python manage.py terraform build` to create the new mappings having templated the terraform module with the new kinesis source.

Contributor guide

Open the contributing guide

Research direction

Start with the kinesis_events Terraform module and the sources.json configuration, then run `python manage.py terraform build` to inspect the generated mappings. Done means the module creates an event source mapping for each configured Kinesis stream instead of only the primary stream.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python, terraform
Domain
cloud, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.