apache / apache/fluss

[server] Support DescribeTabletServers API for safe scaling and rolling upgrades

Open
#3,570 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Description

### Motivation

When scaling a Fluss cluster, a `TabletServer` must not be removed while it still hosts replicas. Since terminating a non-empty server causes under-replication or data unavailability.

Operational tooling (e.g, Fluss Kubernetes Operator, see the FIP discussion thread: https://lists.apache.org/thread/p5sldj4v79t1p6yw5dr52b2j3579df5r) needs to verify that a target server is empty before allowing its termination, and to report per-server tablet load in cluster status.

Today this information is coordinator-internal only:

- `CoordinatorContext.replicasOnTabletServer()` knows the answer, but no client-visible API exposes it.
- `Admin.getServerNodes()` lists tablet servers without their replica load, and `Admin.getClusterHealth()` (#3399) returns only cluster-wide aggregates.

An external controller therefore cannot distinguish an evacuated server from one still hosting replicas.

### Proposed Solution

Add a read-only `DescribeTabletServers` RPC to the Coordinator, following the exact pattern of `GetClusterHealth` (#3399 / PR #3400):

- New proto request/response messages.
- New method on `AdminReadOnlyGateway`, handled by `CoordinatorService`.
- A coordinator handler that tallies replicas per server from in-memory `CoordinatorContext` state.
- A new `Admin.describeTabletServers()` (`@PublicEvolving`) returning, per `TabletServer`, the same four counters getClusterHealth() reports cluster-wide, scoped to that server: numReplicas, inSyncReplicas, numLeaderReplicas, activeLeaderReplicas. These back the operator's `serverGreen(serverId)` predicate (rolling-upgrade gate), the scale-in safety gate (numReplicas == 0), and per-server tablet load in cluster status.

This is a small, low-risk read API: computed from existing in-memory coordinator state, no changes to the network protocol beyond the new RPC, no log/KV format changes, and no new write paths.

This is the only server-side dependency of the Kubernetes Operator FIP's v1alpha1 milestone.

### Willingness to contribute

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the GetClusterHealth pattern referenced in the issue through AdminReadOnlyGateway and CoordinatorService, then inspect CoordinatorContext.replicasOnTabletServer() and Admin.getServerNodes(). Add the DescribeTabletServers request, response, handler, and Admin method so each TabletServer exposes the four scoped counters; done means callers can identify empty and healthy servers without changing write paths or storage formats.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.