apache / apache/iggy

Kafka gateway: Kafka Fetch API (key 1) — bridge to Iggy offset-based polling

Open
#3,536 0 comments 0 reactions 1 assignee Claimed by @ryerraguntla View on GitHub
gateways
Dominant language
Rust
Stars
4.9k
Forks
432
Avg merge
2d 10h
Merged PRs (30d)
173

Description

Part of #3560.

## Summary

Wire Kafka **Fetch (API key 1)** to `IggyBridge::fetch()` — return messages from Iggy by partition offset without consumer groups (Phase 1 assign/manual offset).

## Motivation

Enables simple consumers and `kafka-console-consumer.sh` without `--group` for Phase 1 MVP.

## Depends on

- #3421
- #3533 — IggyBridge core
- #3535 — Produce path (for E2E data)

## Scope

- [ ] Decode Fetch v4–v12 (scoped range)
- [ ] Map fetch offset + max bytes → Iggy poll/fetch by offset
- [ ] Encode RecordBatch response (opaque bytes from Iggy payload)
- [ ] High-watermark / log end offset in response
- [ ] Integration test: produce then fetch same partition/offset

## Acceptance criteria

- [ ] `kafka-console-consumer.sh --from-beginning --partition 0` works **without** consumer group (Phase 1)
- [ ] Offset progression matches Iggy partition offsets

## References

- Rollout plan: https://github.com/apache/iggy/discussions/3253#discussioncomment-17181030

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.