apache / apache/auron

Proposal: Flink support

Open
#1,264 18 comments 11 reactions 1 assignee Claimed by @Tartarus0zm View on GitHub
flink
Dominant language
Rust
Stars
1.8k
Forks
241
Avg merge
2d 12h
Merged PRs (30d)
21

Description

Auron just support Spark(for batch) now. But in terms of capability, it can be used to support Flink (for streaming).

For many user scenarios in Flink, processing at the second or even minute level is perfectly acceptable. Examples include mini-batch processing for unbounded aggregates or minute-level window aggregations.

Auron will build Flink capabilities in three phases.
Phase One: Stateless Operator Support(Calc Operator)
* Support for various operator expressions: such as + - * /
* Support for Flink built-in functions
* Support for user-defined Flink functions

Phase Two: Stateful Operator Support(Agg Operator)

* Support for Flink built-in agg functions
* Support for user-defined Flink agg functions
* Support for flink count distinct

Phase Three: Stateful Operator Support(Join Operator)

* Support Regular Join
* Support Window Join

Supporting Flink in Auron is a massive and complex undertaking. We welcome community members to collaborate and discuss together.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.