[kafka] Integrate Produce with native Fluss append
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Purpose
Part of #4185. PR07 of the Kafka compatibility sequence.
Wire the Produce handler and raw/string transcoder to the TabletServer append gateway. Apply the same table admission contract as Metadata, isolate conversion and append errors per partition, map acknowledgements and offsets, and register Produce v3-v11.
### Acceptance criteria
- [ ] Only compatible pre-created tables and valid bucket IDs reach append.
- [ ] One invalid table or partition does not suppress independent valid writes.
- [ ] Gateway acks/timeout and partition offsets/replication errors map correctly.
- [ ] Kafka calls tryCompleteActions immediately after synchronous produceLog invocation, including throws, without waiting for async acknowledgements.
- [ ] A real Kafka producer discovers native DDL tables and writes raw/string records with acks=0/1/all; native Fluss readback verifies data and metadata.
### Dependencies
PR06 raw/string conversion (#4282) and independent core PR #3412 / #3455. Core changes remain owned by #3412.
### Scope boundary
No new core implementation, Kafka table creation, JSON, authentication, idempotence or transactions.
### Implementation
Draft PR #4286: https://github.com/apache/fluss/pull/4286
Java 11 validation: 82 unit tests and 4 integration tests passed at `dc5fb6599b8c5e155cb17280df71a74ee34c9ffa`. Review and merge remain pending.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the Produce handler, raw/string transcoder, and TabletServer append gateway described in the issue; compare draft PR #4286 and its listed validation commit, including Metadata's table-admission contract. Done means the acceptance criteria pass, including isolated partition errors, mapped acknowledgements and offsets, immediate completion behavior, and real Kafka/native Fluss integration validation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems, stream-processing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 20/100