Graylog2 / Graylog2/graylog2-server
Allow two Graylog clusters to read from same Kinesis steam
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
## What?
Allow two different Graylog clusters to use the AWS Kinesis CloudWatch Input to read from the same Kinesis stream simultaneously.
## Why?
I am starting a project to migrate from Graylog V3.x to Graylog 5.x clusters. My plan is to run both clusters in parallel so users can use both while they transition all of their Alerts/Dashboards/etc from old to new.
I would like the ability to have an Input on both clusters that read from the same Kinesis stream. As it sits today, it appears that I would have to disable the Input on the old cluster in order for the new cluster to see new messages on the stream as they both use the same DynamoDB table to track where they are in reading data in the stream.
For example, the stream name is eventflow and the DynamoDB table name (for both) is graylog-aws-plugin-eventflow.
Seems like the solution would be to make the DynamoDB table name unique for each Graylog cluster?
Please advise if there is already a way to do this and I have overlooked how to do so.
## Your Environment
* Graylog Version: 5.2.3
* OpenSearch Version: 2.11.1
* MongoDB Version: 6.0.12
* Operating System: Amazon Linux 2
* Browser version: NA
Contributor guide
Assessment
This issue has not been assessed yet.