opensearch-project / opensearch-project/data-prepper
Provide option to use expressions in S3 sink bucket_owners config
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
S3 sink currently supports the following config where the bucket name and account id are static
bucket_owners:
my-bucket-01: 123456789012
my-bucket-02: 999999999999
The feature request is to allow dynamic values in the bucket name and accountids
Describe the solution you'd like
Extend the config to allow Data Prepper expressions in the above config like
bucket_owners:
"{/bucket-field1}": "{/account-field1}"
"{/bucket-field2}": "{/account-field2}"
Describe alternatives you've considered (Optional)
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the S3 sink's bucket_owners configuration handling and trace how Data Prepper expressions are evaluated in sink settings. Confirm that both bucket names and account IDs can resolve dynamically from event fields, and verify the requested configuration examples work end to end.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, java
- Domain
- backend, cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100