Kafka gateway: Kafka gateway IggyBridge core — SDK client and stream/topic mapping
- Dominant language
- Rust
- Stars
- 4.9k
- Forks
- 432
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 173
Description
Part of #3560.
## Summary
Implement the `IggyBridge` integration layer in `gateways/kafka/` that connects decoded Kafka protocol handlers to the Iggy Rust SDK.
## Motivation
Phase 1 of the [Kafka → Iggy bridge rollout](https://github.com/apache/iggy/discussions/3253#discussioncomment-17181030) requires mapping Kafka topics/partitions to Iggy streams/topics and exposing produce/fetch/watermark operations. This module is shared by all Phase 1 API handlers.
## Depends on
- #3421 — TCP listener, request decode, ApiVersions firewall
## Scope
- [x] `bridge/iggy_bridge.rs` (or equivalent) with async Iggy TCP client lifecycle
- [x] `ensure_stream_and_topic(kafka_topic, partition_count)` → Iggy stream + topic create-if-missing
- [x] Topic name → Iggy stream/topic mapping config (TOML/env)
- [x] Connection config: Iggy address, credentials (`SecretString` pattern)
- [x] `high_watermark(stream, topic, partition)` for ListOffsets
- [x] Error mapping: Iggy errors → Kafka error codes where applicable
- [x] Unit tests with mock or harness; no panics on Iggy unreachable at handler boundary
## Out of scope
- Produce/Fetch handler wiring (separate issues)
- Consumer group offset storage
## Acceptance criteria
- [x] Bridge module compiles and is invoked from integration test stub
- [x] `ensure_stream_and_topic` idempotent on repeated calls
- [ ] README section documents mapping rules and config
## References
- Rollout plan: https://github.com/apache/iggy/discussions/3253#discussioncomment-17181030
- Parent discussion: https://github.com/apache/iggy/discussions/3253
- Related: https://github.com/apache/iggy/discussions/3081
- Prior issue: #3421
Contributor guide
Research direction
Start with gateways/kafka/bridge/iggy_bridge.rs and the integration test stub to verify the implemented mapping and configuration behavior. Check the repository README structure, then document the topic-to-stream/topic mapping rules and TOML or environment configuration; done means the README section is added and the existing tests still pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems, documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100