[improvement] consolidate the kinesis_streams and kinesis_events terraform modules
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 324
- PR merge metrics
- No merged PRs in 30d
Description
## Background
There are currently two different terraform modules related to kinesis:
[tf_stream_alert_kinesis_events](https://github.com/airbnb/streamalert/tree/277d3b9664441feb2b2037055c101e54fb30c676/terraform/modules/tf_stream_alert_kinesis_events)
[tf_stream_alert_kinesis_streams](https://github.com/airbnb/streamalert/tree/277d3b9664441feb2b2037055c101e54fb30c676/terraform/modules/tf_stream_alert_kinesis_streams)
### Description
It is unclear why these should be separate, since they are very much so dependent on each other.. For example, the `kinesis_events` module cannot be enabled if the `kinesis_streams` module is not enabled. This adds extra unnecessary complexity and can easily lead to a misconfigured cluster, resulting in terraform errors (having `kinesis_events` without `kinesis_streams` breaks things badly).
## Desired Change
- Consolidate all of the code in the above modules into one module.
- Update kinesis [streams](https://github.com/airbnb/streamalert/blob/277d3b9664441feb2b2037055c101e54fb30c676/stream_alert_cli/terraform/kinesis_streams.py#L18)/[events](https://github.com/airbnb/streamalert/blob/277d3b9664441feb2b2037055c101e54fb30c676/stream_alert_cli/terraform/kinesis_events.py#L18) terraform generation code.
- Ensure the [CLI](https://git.musta.ch/csirt/streamalert/blob/1177e194fdd3bc0453dcc17dacc1e88e0e7fc307/stream_alert_cli/kinesis/handler.py#L23) handler for kinesis is working/compliant with above changes.
- Update unit tests.
Contributor guide
Research direction
Start with terraform/modules/tf_stream_alert_kinesis_events and terraform/modules/tf_stream_alert_kinesis_streams, then inspect stream_alert_cli/terraform/kinesis_streams.py, kinesis_events.py, and kinesis/handler.py. Update the related unit tests after consolidating the modules. Done means the Terraform generation and CLI handler work with one module and the separate-module misconfiguration is no longer possible.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python, terraform
- Domain
- cli, cloud, infrastructure, testing
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100