[server] Support dynamic rebalance concurrency control
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
### Motivation
The rebalance manager currently admits one bucket-level task at a time and waits for that task to finish or time out before starting the next one. This preserves ordering but makes large rebalance plans unnecessarily slow, especially when most bucket migrations complete quickly and the cluster has enough capacity to process several independent migrations safely.
### Proposed solution
Introduce a dynamically configurable maximum number of in-flight bucket-level rebalance tasks, for example `coordinator.rebalance.max-inflight-tasks`:
- Keep the default value at 1 for backward-compatible behavior.
- Admit up to the configured number of independent bucket tasks.
- Allow runtime configuration updates through the coordinator event thread.
- Setting the value to 0 pauses admission of new tasks without canceling tasks already running or being reconciled.
- Reducing the limit does not abort admitted tasks; it only delays further admission until the number of running tasks falls below the new limit.
- Preserve per-attempt fencing so timeout, late NotifyLeaderAndIsr responses, cancellation, and coordinator restart cannot complete the wrong task.
- Exclude soft-timed-out tasks from normal admission slots while continuing bounded reconciliation.
The concurrency implementation should build on the recoverable timeout and event-fencing state machine tracked by #3859. The timeout recovery change must not depend on this concurrency feature.
Related pull request: #3537.
### Acceptance criteria
- The default configuration still executes one bucket task at a time.
- A configured limit of N never admits more than N normal in-flight tasks.
- Runtime increases, decreases, and pause/resume updates are applied safely.
- Completion or timeout of one task admits the next eligible task without exceeding the current limit.
- Cancel and coordinator recovery preserve task identity and do not duplicate or lose work.
- Tests cover dynamic reconfiguration, stale events, timeout interaction, cancellation, and restart recovery.
### Non-goals
This task does not change rebalance plan storage or replica reassignment semantics.
### Willingness to contribute
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.