feat(connector): Add Opensearch source connector
- Dominant language
- Rust
- Stars
- 4.9k
- Forks
- 432
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 173
Description
### Description
Add a Opensearch source connector that polls documents from a Opensearch index and produces JSON messages into Iggy.
Users should be able to ingest Opensearch documents into Iggy streams with stable cursor-based polling and persisted connector state.
### Affected area / component
Connectors
### Proposed solution
Scope
Add the source connector crate and sample runtime config.
Document source configuration, retry behavior, and primary-key cursor requirements.
Add Docker-backed integration coverage for producing indexed documents.
### Alternatives considered
_No response_
### Contribution
- [ ] I'm willing to submit a pull request to implement this feature
### Good first issue
- [ ] I think this could be a good first issue for a new contributor
Contributor guide
Research direction
Begin in the Connectors area by comparing existing source connectors and their runtime configuration patterns. Add the source connector crate and sample configuration, document source settings, retries, and primary-key cursor requirements, then run the Docker-backed integration coverage to verify indexed documents are produced into Iggy with persisted state.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, rust
- Domain
- data-engineering, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100