[Improve][Zeta] Add historical task metrics persistence and time-range query support
- Dominant language
- Java
- Stars
- 9.7k
- Forks
- 2.4k
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 204
Description
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues) and found no similar feature request.
### Motivation
SeaTunnel already has runtime metrics and realtime observability for running jobs, but it does not yet provide a complete mechanism for **historical task metrics persistence** and **time-range querying**.
At the moment, the gap is mainly around questions like:
- How many records were read or written during a specific time range?
- What was the throughput trend of a source / transform / sink task over time?
- What sampling interval is used for metric collection?
- Where are these sampled metrics persisted for later analysis?
This is especially important for long-running streaming and CDC jobs, where users often need to investigate performance regressions, backpressure, throughput drops, and task-level bottlenecks after the fact.
### Current limitation
From the current implementation, SeaTunnel Zeta mainly provides:
- latest task metrics snapshots
- short-window realtime observability
- final job metrics after job completion
However, it does **not** provide a built-in solution for:
- persistent storage of task/operator historical metrics
- configurable sampling interval for historical collection
- querying metrics by time range
- a clear storage abstraction for historical metrics backend
- REST/UI support for historical metric exploration
### Expected behavior
It would be useful to support a historical metrics mechanism for Zeta, for example:
1. Configurable metric sampling interval
2. Configurable retention period
3. Persistent storage backend for historical metrics
4. Query API by:
- job
- pipeline
- task / vertex / operator
- metric name
- time range
5. Optional UI support for trend charts and time-series inspection
### Possible design direction
A possible direction could be:
- keep the current realtime observability path lightweight
- introduce a separate historical metrics export/persistence path
- support pluggable backends such as:
- Prometheus-compatible remote storage
- OpenTelemetry metrics pipeline
- InfluxDB
- other external TSDB systems
- avoid coupling historical metrics storage with checkpoint state
This would keep checkpoint/recovery semantics clean while making observability more production-friendly.
### Use cases
- Investigate how many records were read in the last 10 minutes / 1 hour
- Compare source and sink throughput over a time range
- Analyze historical backpressure trends
- Troubleshoot intermittent performance issues after the job has recovered or restarted
- Build dashboards and alerting on SeaTunnel task-level metrics
### Looking for contributors
I believe this would be a valuable improvement for SeaTunnel Zeta observability.
If the community agrees with this direction, I would like to ask:
- Is there already a related design or roadmap?
- Would maintainers be open to a proposal for this feature?
- Are there contributors interested in collaborating on the design and implementation?
Contributor guide
No contributing guide indexed for this repository
Research direction
No files, tests, or entry points are named in the issue. Start by reviewing SeaTunnel Zeta's existing runtime metrics and realtime observability paths, then define the sampling, retention, persistence, time-range query, and REST/UI boundaries. Done means the community has an agreed design and implementation scope for historical task metrics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, prometheus
- Domain
- backend, distributed-systems, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100