opensearch-project / opensearch-project/data-prepper
[BUG] Disabling and Enabling DynamoDB streams requires a restart of Data Prepper
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Describe the bug
If I disable and enable the streams on my DynamoDB table, Data Prepper will continue looking for the old stream ARN that does not exist, and does not do a rediscovery of the stream ARN.
To Reproduce
Steps to reproduce the behavior:
- Start a DynamoDB pipeline reading from streams
- Disable and re-enable DynamoDB streams on the table
- Observe the error from Data Prepper that the stream does not exist
Expected behavior
Data Prepper could gracefully handle the case where the stream does not exist, and rediscover the new Stream ARN
Additional context
Add any other context about the problem here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test is named in the issue. Start by reproducing the DynamoDB pipeline failure after disabling and re-enabling streams, then trace stream discovery and error handling; done means the new stream ARN is rediscovered without restarting Data Prepper.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, java
- Domain
- stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100