apache / apache/fluss

Add completion timestamp and historical listing to RebalanceProgress

Open
#3,965 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Motivation

`Admin#listRebalanceProgress` returns `RebalanceProgress`, which carries `rebalanceId`, `status`, a `progress` fraction, and a per-bucket map — but no timestamp for when a rebalance started or finished. In addition, calling `listRebalanceProgress(null)` only ever returns the currently in-progress rebalance task; there's no way to retrieve information about rebalances that have already completed.

This makes it impossible to build any operator-facing tooling that answers "when did the cluster last rebalance?" or "how long did the last rebalance take?" — useful for health dashboards, alerting, and audit trails. Today that information is only recoverable by tailing server logs.

### Solution

- Add `startedAt`/`completedAt` (or equivalent) timestamp fields to `RebalanceProgress`.
- Retain a bounded history of recently-completed rebalance tasks (server-side or ZooKeeper-backed) so that `listRebalanceProgress` (or a new method) can return past tasks, not just the current in-progress one — including their final status (COMPLETED/FAILED/CANCELLED) and timestamps.
- A retention window/limit (e.g. keep the last N completed tasks) would bound the added storage/memory cost.

### Anything else?

_No response_

### Willingness to contribute

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with Admin#listRebalanceProgress and RebalanceProgress, then trace how listRebalanceProgress(null) finds the current task and where rebalance state is stored. Define the API shape, timestamp semantics, completed-task retention limit, and final statuses before identifying the server-side or ZooKeeper-backed implementation. Done means current and bounded historical rebalances can be listed with timestamps and final status.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.