[Feature][Zeta] Add a dedicated backpressure diagnostics view for running jobs
- Dominant language
- Java
- Stars
- 9.7k
- Forks
- 2.4k
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 204
Description
## Background
SeaTunnel already exposes some backpressure-related realtime signals, and the current UI can color edges with blocked ratio and queue fill ratio.
That is a good starting point, but it is still not enough for real backpressure diagnosis.
In practice, when a pipeline is slow, users do not just need to know that "an edge looks red". They need to know:
- which edge is the worst bottleneck right now
- whether the bottleneck is stable or transient
- whether it is caused by source production speed, transform CPU cost, sink write/commit pressure, or downstream queue saturation
- whether the pressure is getting worse or recovering
## Problem to solve
Backpressure information currently exists mostly as low-level samples. SeaTunnel still lacks a dedicated user-facing view that turns those samples into an actionable diagnosis workflow.
This leads to slow troubleshooting and makes it hard to compare multiple blocked paths in the same job.
## Proposed scope
Add a dedicated backpressure diagnostics view for running jobs.
Suggested first-delivery scope:
- rank the most blocked edges and downstream bottleneck vertices in one place
- show queue size, queue capacity, fill ratio, blocked ratio, and blocked duration trend together
- distinguish between chronic pressure and short spikes
- provide a direct path from the backpressure panel into the related vertex/edge details and logs
- make the view work together with the runtime graph, but keep it independently useful as a focused bottleneck page
## Why this is not just a UI cleanup
A useful backpressure view may require new or clarified contracts for:
- how blocked time is sampled and aggregated
- whether queue metrics are instantaneous, rolling-window, or max/min summaries
- how long samples are retained
- how high-frequency metrics are cleaned up after job finish/failure
- how to avoid high write amplification for observability-only state
## STIP requirement before implementation
Because this feature may extend realtime metric contracts and has non-trivial runtime-cost implications, **the claimant should submit a STIP design first and get maintainer agreement before starting implementation**.
The STIP should clarify:
- the exact metric model used by the backpressure view
- whether existing metrics are sufficient or new metrics are required
- how the view handles large jobs and fast-refresh scenarios
- how backpressure diagnosis should interact with checkpoint and sink-commit signals
- which data is ephemeral observability state versus stable user-facing API output
## Acceptance criteria
- Users can quickly identify the top blocked edges and likely bottleneck vertices for a running job.
- The view shows both current state and short-term trend, not just a single latest value.
- Data definitions are documented clearly enough that users do not have to infer meaning from implementation details.
- English and Chinese docs are updated.
## Non-goals for the first version
- full root-cause automation for every slowdown
- storage of long-term historical backpressure data
- connector-specific tuning recommendations in the first delivery
## Related work
- PR #11244
- Issue #11351
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing PR #11244 and Issue #11351, then prepare the required STIP design before implementation. Define the metric model, retention and refresh behavior, runtime-cost safeguards, and interactions with checkpoint and sink-commit signals. Done means the agreed view meets the acceptance criteria and English and Chinese documentation is updated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, frontend, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100