apache / apache/wayang

Restructure the Kafka related operators

Open
#469 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
274
Forks
140
Avg merge
5d 16h
Merged PRs (30d)
4

Description

Proposal:

We create a **wayang-kafka** module inside of **wayang-platforms**.
This leads to a cleaner approach to add these operators.

Right now we edit both **wayang-java** and **wayang-spark**, where both of them contain versions of operators related to reading and writing to Kafka.

Conceptually, having all implementations for the specific APIs in **wayang-kafka** would be neat.

Then you could have wayang-kafka/src/main/java/JavaKafkaOperator wayang-kafka/src/main/java/SparkKafkaOperator etc. and the respective platforms themselves don't have to concern about Kafka.

It is planned to implement this as soon as the initial demos for Kafka integration are ready.

Target: Release 1.1 - Q1-2025.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.