apache / apache/iggy

feat(connector): Add Opensearch sink connector

Open
#3,504 3 comments 0 reactions 0 assignees View on GitHub
connectors
Dominant language
Rust
Stars
4.9k
Forks
432
Avg merge
2d 10h
Merged PRs (30d)
173

Description

### Description

Add a Opensearch sink connector that consumes Iggy messages and indexes them into a Opensearch index through the official Rust SDK.

### Motivation
Users should be able to route Iggy stream data into Opensearch for full-text search and retrieval without writing a custom connector.

### Scope
- Add the sink connector crate and sample runtime config.
- Document sink configuration and payload behavior.
- Add Docker-backed integration coverage for indexing JSON messages.

### Affected area / component

Connectors

### Proposed solution

_No response_

### Alternatives considered

_No response_

### Contribution

- [x] I'm willing to submit a pull request to implement this feature

### Good first issue

- [x] I think this could be a good first issue for a new contributor

Contributor guide

Open the contributing guide

Research direction

Start in the Connectors area and review how existing sink connectors are structured before checking the official Rust SDK and the sample runtime configuration. Done means the connector indexes JSON Iggy messages into an OpenSearch index, its sink configuration and payload behavior are documented, and Docker-backed integration coverage passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, rust
Domain
data-engineering, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.