[Feature][Zeta] Add a worker node resource view to the Zeta Web UI
- Dominant language
- Java
- Stars
- 9.7k
- Forks
- 2.4k
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 204
Description
## Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct).
## Search before asking
- [x] I searched the existing [feature requests](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no issue proposing a worker/node resource view. #11351/#11352/#11353 cover job- and pipeline-level observability, not per-node resource usage.
## Background
Zeta's REST API already returns rich per-node monitoring data via `GET /system-monitoring-information`: roughly 35 fields including CPU load (process/system/systemAverage), heap/physical/managed memory, GC counts and time, and thread counts (the full payload is already modeled in `seatunnel-engine-ui/src/service/manager/types.ts`). However, the Workers/Master page (`seatunnel-engine-ui/src/views/managers/index.tsx`) only renders 4 of those fields today - Host, Port, Physical MEM Total, and Heap MEM Used - and an Action column is present in the source but commented out.
Separately, Zeta already computes a full task-to-worker assignment (`GET /trace/task-mapping/:jobId`, backed by `TaskMappingBuilder`) with per-task host/worker binding, but that endpoint has no UI consumer at all today.
What is genuinely missing, not just unrendered, is per-worker **slot accounting**: `OverviewInfo` only exposes cluster-wide `totalSlot`/`unassignedSlot` integers; there is no per-node slot total/used breakdown anywhere in the current model.
In Flink's Web UI, the "Task Managers" tab is the equivalent first-class page: per-TM memory, CPU, GC, and slot occupancy, plus the tasks currently scheduled on it.
## Problem to solve
Job-level and pipeline-level observability (#11351/#11352/#11353) can tell you *that* a vertex is slow, but not *why* at the infrastructure layer - e.g. whether the host running that task is CPU-saturated, low on heap, or has no free slots left for retry/redeploy. Today that requires shelling into a node directly, even though most of the raw data is already being collected by the cluster itself.
## Proposed scope
Extend the existing Workers/Master page into a proper node resource view, mostly by surfacing data that already exists:
- render the full already-fetched `/system-monitoring-information` payload (CPU load, GC, thread count, etc.) instead of the current 4 columns
- add per-worker slot accounting (total/used) to the node model and surface it in the same view - this is the one part that needs new server-side bookkeeping, since slots are currently only tracked cluster-wide
- wire the existing `/trace/task-mapping` endpoint into this page so a worker row can expand into "tasks currently running here", linking back to the owning job
- keep this primarily a UI + light REST-extension change, since the bulk of the raw resource data is already collected
## Why this should be a dedicated feature
This is cluster/infrastructure-level observability, distinct from job/pipeline-level observability:
- it is scoped by node, not by job or vertex, so it needs its own UI treatment and a small REST extension (per-worker slots)
- it must keep working (and stay useful) even when no job is currently running
- per-worker slot accounting must stay consistent with the live scheduler view, which is a correctness-sensitive contract, not just a display change
## STIP requirement before implementation
Because this adds new per-worker slot-accounting state, **the claimant should submit a STIP design first and get maintainer agreement before starting implementation**. The STIP can stay narrow given how much already exists:
- how per-worker slot total/used is derived and kept consistent with the scheduler's live view
- whether `/trace/task-mapping` needs any changes to serve a per-worker "tasks running here" list efficiently, or is already sufficient as-is
- refresh interval and cost on larger clusters for the now-fuller `/system-monitoring-information` payload
## Acceptance criteria
- The Workers/Master page renders the full monitoring payload already returned by `/system-monitoring-information`, not just today's 4 fields.
- Users can see per-worker slot usage and the tasks currently assigned to each worker, without SSHing into a host.
- The view remains useful when the cluster is idle (no jobs running).
- REST/API and UI behavior are documented clearly.
- English and Chinese docs are updated.
## Non-goals for the first version
- historical/long-term retention of node resource metrics
- cluster capacity planning or autoscaling recommendations
- per-node log tailing (already covered by the existing worker log viewer)
## Related work
- Issue #11351
- Issue #11352
- Issue #11353
## Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.