apache / apache/iggy

Kafka gateway: Kafka Produce API (key 0) — bridge to Iggy send_messages

Open
#3,535 0 comments 0 reactions 1 assignee Claimed by @ryerraguntla View on GitHub
gateways
Dominant language
Rust
Stars
4.9k
Forks
432
Avg merge
2d 10h
Merged PRs (30d)
173

Description

Part of #3560.

## Summary

Wire Kafka **Produce (API key 0)** to `IggyBridge::produce()` — decode RecordBatch payloads and send to Iggy with balanced partitioning.

## Motivation

Core write path for Kafka migration: existing producers must work without code changes.

## Depends on

- #3421
- #3533 — IggyBridge core

## Scope

- [ ] Decode Produce v3–v9 (scoped range); preserve RecordBatch as opaque bytes where possible (hot path)
- [ ] Map Kafka topic + partition → Iggy stream/topic/partition
- [ ] Partitioning: balanced / keyed per Kafka record (document semantics)
- [ ] Produce response with correct base offset / error per partition
- [ ] KAFKA-18659: ApiVersions advertises Produce min=0 for librdkafka compatibility (encode layer)
- [ ] Integration test: produce via gateway → verify message in Iggy (SDK or CLI)

## Acceptance criteria

- [ ] `kafka-console-producer.sh` or `kcat -P` can send to a topic through the gateway
- [ ] Messages visible in Iggy stream/topic
- [ ] At-least-once documented (Phase 1)

## References

- Rollout plan: https://github.com/apache/iggy/discussions/3253#discussioncomment-17181030

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.