apache / apache/iggy

feat: OTLP Sink Connector

Open
#3,526 3 comments 0 reactions 0 assignees View on GitHub
connectors
Dominant language
Rust
Stars
4.9k
Forks
432
Avg merge
2d 10h
Merged PRs (30d)
173

Description

### Description

Add a native OTLP (OpenTelemetry Protocol) sink connector for Apache Iggy using the opentelemetry-otlp Rust crate.

This connector will enable streams stored in Iggy to be exported to OpenTelemetry-compatible observability platforms such as Observe, Datadog, New Relic, Grafana Cloud, Honeycomb, Elastic, Splunk, and any backend supporting OTLP over gRPC or HTTP.

The OTLP sink is a foundational building block for positioning Iggy as a high-performance ingestion, buffering, replay, and routing layer for observability, AI telemetry, and event-driven architectures.

Connector Type: Sink Connector
---
Iggy Stream

OTLP Sink

OTLP Endpoint
---

Initial version:
- Logs

Future enhancements:
- Metrics
- Traces

===
Metrics
----
Expose connector metrics:
- otlp_sink_messages_sent_total
- otlp_sink_messages_failed_total
- otlp_sink_batches_sent_total
- otlp_sink_batch_latency_ms
- otlp_sink_retry_count_total

### Affected area / component

Connectors

### Proposed solution

_No response_

### Alternatives considered

_No response_

### Contribution

- [ ] I'm willing to submit a pull request to implement this feature

### Good first issue

- [x] I think this could be a good first issue for a new contributor

Contributor guide

Open the contributing guide

Research direction

Start in the Connectors area by locating existing sink connectors and reviewing how they handle delivery, batching, retries, and connector metrics. Then read the opentelemetry-otlp Rust crate documentation and define the initial logs-only scope; done should include export to an OTLP endpoint over gRPC or HTTP and the listed sink metrics.

Written by the indexing model from the issue text.

Assessment

Tech stack
grpc, rust
Domain
observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.