Support Kafka Protocol 4: Add utils to convert Kafka MemoryRecords
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/alibaba/fluss/issues) and found nothing similar.
### Motivation
Parent issue: https://github.com/alibaba/fluss/issues/486
As the Fluss protocols is different with Kafka protocol, so for the purpose of compatibility, we need a util to convert Kafka MemoryRecords to LogRecordBatch, and convert LogRecordBatch to Kafka MemoryRecords.
The util should be like:
```java
package com.alibaba.fluss.kafka;
import com.alibaba.fluss.record.LogRecordBatch;
import org.apache.kafka.common.record.MemoryRecords;
public class RecordsUtil {
public static LogRecordBatch convert(MemoryRecords records) {
return null;
}
public static MemoryRecords convert(LogRecordBatch batch) {
return null;
}
}
```
### Solution
_No response_
### Anything else?
_No response_
### Willingness to contribute
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with parent issue #486 and the proposed com.alibaba.fluss.kafka.RecordsUtil API, then inspect LogRecordBatch and Kafka's MemoryRecords definitions to determine the required mappings. Done means both conversion directions support the compatibility use case and are covered by verification for representative records.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kafka
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100