apache / apache/fluss

Refactor the RPC MetadataResponse to introduce invalid partition set

Open
#2,202 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Description

Currently, when a `MetadataRequest` is sent, if the requested tableBucket contains partitions that do not exist, fluss server will directly throws an RPC-level exception: `PartitionNotExistException`. This makes it impossible for the client to handle the error gracefully. For example, the `LogFetcher` would crash immediately, affecting the reading of other normal tableBuckets.

To address this, it is proposed to return a set of invalid partitions (i.e., all non-existent partitions) instead of throwing an exception. This allows the `LogFetcher` to report these invalid partitions to the user, who can then customize the next fetch logic (e.g., skip or retry specific partitions).

### Willingness to contribute

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Trace the MetadataRequest handling and the LogFetcher behavior described in the issue. Determine how the MetadataResponse should represent non-existent partitions without an RPC-level exception, then verify that invalid partitions can be reported while other normal tableBuckets continue to be read.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.