apache / apache/fluss

[FIP-28] Smoothly reroute writes with in-flight requests after partition retirement

Open
#4,166 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Description

Parent issue: #3631

Follow-up to #4119 and PR #4120.

A client may resolve an original partition as present, enqueue and send a batch, and then receive an explicit missing-partition response after retention removes that partition. PR #4120 can reroute queued batches when the original target has no in-flight requests. If requests for that target are still in flight, it aborts only the affected target to preserve idempotence and batch-sequence correctness. In a Flink job, that abort causes a failover.

The race is expected to be uncommon, but production historical writes should handle it without a job-level failover. The handoff must account for the fact that the original and shared historical partitions are different physical TableBuckets with independent writer IDs and batch-sequence namespaces.

The implementation should:

- Coordinate with the durable FREEZING and RETIRED lifecycle so rerouting starts only from a deterministic server response.
- Stop draining new batches to the original target while a handoff is in progress.
- Resolve outstanding responses for the original target and distinguish successful, explicitly rejected, and ambiguous requests.
- Preserve ordering and idempotence while moving safe queued batches to the historical target.
- Remove old in-flight sequence state and assign writer IDs and sequences in the historical target namespace without reusing original-target sequence state.
- Keep failure handling scoped to the affected original partition and define the fallback for outcomes that cannot be proven safe to reroute.
- Add concurrency tests covering responses that race with handoff, retries, ambiguous timeouts, and multiple original partitions sharing one historical bucket.

### Willingness to contribute

- [ ] I am willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing PR #4120 and the related issues #3631 and #4119 to understand the existing queued-batch rerouting and partition lifecycle behavior. Trace the client handoff flow described here, then add concurrency coverage for racing responses, retries, ambiguous timeouts, and shared historical buckets; done means safe ordering and idempotence without job-level failover.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.