apache / apache/fluss

Optional channel selection for source split in FlinkSourceEnumerator

Open
#3,269 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Motivation

We'd better to introduce the similar mechanism (shuffle assign without the partition) like [paimon](https://github.com/apache/paimon/blob/959170d8fe1e871f54d17e4f6a6980eee389cf6e/paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/source/ContinuousFileSplitEnumerator.java#L336) did

### Solution

_No response_

### Anything else?

_No response_

### Willingness to contribute

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating FlinkSourceEnumerator and compare its source-split assignment with the referenced Paimon ContinuousFileSplitEnumerator implementation. Clarify how optional channel selection should work and which existing source-enumerator tests cover split assignment. Done means the enumerator supports the requested optional channel-selection behavior with matching test coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
stream-processing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.