opensearch-project / opensearch-project/data-prepper
Securing Sensitive Pipeline Configuration Data
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
Currently, Data Prepper pipeline configurations are loaded from a static plain text file. Sensitive configuration data (ie. usernames and passwords) can be stored in pipeline configurations.
Describe the solution you'd like
I would like to support to load sensitive data from secure locations. Some examples are:
- Docker secrets
- A key manager
- others
Ideally, this feature would be supported through plugins allowing the community to build their own as well.
Describe alternatives you've considered (Optional)
A clear and concise description of any alternative solutions or features you've considered.
Additional context
This came out of a discussion from: #947
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the discussion in #947 and the current static plain-text pipeline configuration loading described here. Identify the extension point for plugin-backed secure locations such as Docker secrets or a key manager; done requires an agreed scope and plugin contract that supports loading sensitive configuration data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100