opensearch-project / opensearch-project/data-prepper
Support proxy in OpenSearch source
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
I have a pipeline that reads from an OpenSearch index and writes in another one. That OpenSearch server is behind a proxy, so I need to configure it in the source and the sink
Describe the solution you'd like
I think the best solution is adding a proxy to the opensearch source:
source:
opensearch:
...
proxy: https://myproxy:8080
Or within the connection field:
source:
opensearch:
...
connection:
proxy: https://myproxy:8080
Describe alternatives you've considered (Optional)
proxyfield in theopensearchsource, or in theconnectionsubfieldproxyfield in thedata-prepper-config.yaml- Environment variable
Additional context
The proxy configuration is already supported for the opensearch sink. It was reported in #300 and implemented in #479
Thank you very much in advance!
I know how hard is maintaining an open source project, so if you need help, I can try to implement this
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by examining the existing OpenSearch sink proxy support referenced in #300 and #479, then compare it with the OpenSearch source and connection configuration. Confirm the intended proxy placement and verify that a source-to-index pipeline can use the proxy while preserving existing sink behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, data-engineering
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100