opensearch-project / opensearch-project/data-prepper
SNS as Sink
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
Pipeline users want to send message to an Amazon Simple Notification Service (SNS) topic.
Describe the solution you'd like
Create a new sink in Data Prepper which outputs data to topic using codec
sink:
- sns:
topic_name: "mytopic"
id: << String>>
aws:
region: us-east-1
sts_role_arn: "arn:aws:sns:us-east-1:1234567:hello"
codec:
ndjson:
Additional context
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing Data Prepper's existing sink implementations and the proposed sink configuration, especially topic_name, AWS region, STS role, and ndjson codec settings. Use the linked AWS SNS Java SDK example to understand the integration. Done means pipeline data can be sent to a configured SNS topic with the requested codec and AWS options.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, java
- Domain
- backend, cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100