feat(connectors): add Redis sink connector for Iggy streams
- Dominant language
- Rust
- Stars
- 4.9k
- Forks
- 432
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 173
Description
### Description
## Summary
Add a new Rust sink plugin (`redis_sink`) to the Iggy connectors runtime that consumes messages from Iggy streams/topics and writes them to Redis.
## Motivation
Redis is widely used as a low-latency cache, materialized view store, and event buffer (via Redis Streams). A native Redis sink enables pipelines such as:
- Iggy → Redis cache/materialized state
- Iggy → Redis Streams for downstream fan-out
- Event archival with optional TTL for cache use cases
There is currently no Redis sink in `core/connectors/sinks/`.
## Scope (v1)
Implement an initial sink with **Redis Streams** mode:
- `XADD` writes with pipelined batching
- Deterministic entry IDs for idempotent replay (pattern: `mongodb_sink` / `surrealdb_sink`)
- Configurable `connection_url` (`SecretString`), `stream_key`, `batch_size`, `payload_format`
- Optional Iggy metadata fields (`stream`, `topic`, `partition`, `offset`, `message_id`, headers)
- Transient vs permanent error mapping (connection errors retry; `WRONGTYPE`, auth failures do not)
- `open()` connectivity check (`PING`), `close()` stats logging
### Affected area / component
_No response_
### Proposed solution
### Suggested config (v1)
```toml
type = "sink"
key = "redis"
path = "target/release/libiggy_connector_redis_sink"
[plugin_config]
connection_url = "redis://localhost:6379"
mode = "stream"
stream_key = "iggy:events"
batch_size = 100
payload_format = "json"
include_metadata = true
max_retries = 3
Contributor guide
Assessment
This issue has not been assessed yet.