Add Prometheus metrics and Grafana dashboards for workflow execution observability
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
### Feature Summary
Texera's Amber engine already collects rich runtime statistics internally (tuple counts, processing times, worker counts, execution duration) via `StatisticsManager` and `ExecutionStatsService` — but these are only visible in the UI during an active session and aren't available for alerting, historical analysis, or system-level correlation.
**This feature adds:**
- A **Prometheus `/metrics` endpoint** on the coordinator service exposing workflow, operator, and JVM metrics
- **Pre-built Grafana dashboards** for workflow overview, operator performance, and system health
- **Docker Compose integration** (opt-in `observability` profile) for one-command setup of Prometheus + Grafana
**Problems this solves:**
- No way to monitor execution health outside the UI
- No alerting for stalled workflows or resource exhaustion
- No historical trend analysis for capacity planning
- JVM/system metrics (heap, GC, threads) are completely invisible
### Proposed Solution or Design
**Instrument the existing stats pipeline** with [Micrometer](https://micrometer.io/) (JVM metrics facade) and expose a standard Prometheus scrape endpoint. No changes to the existing stats interfaces — just an additional export path.
**High-level approach:**
1. Add Micrometer & Prometheus registry dependency to `amber/build.sbt`
2. Create a `MetricsService` singleton wiring existing `StatisticsManager` / `ExecutionStatsService` data into Micrometer meters
3. Expose `GET /api/metrics` via a new Dropwizard resource
4. Add Prometheus + Grafana to `bin/single-node/docker-compose.yml` under an optional `observability` profile
5. Ship 3 pre-provisioned Grafana dashboards (Workflow Overview, Operator Deep-Dive, System Health)
**Key metrics:**
- Workflow: active count, execution duration, state transitions
- Operator: input/output tuple rates, processing time breakdown, worker count
- System: JVM heap, GC pauses, thread count (via Micrometer's built-in JVM binders)
**Contributors:** We (@musxeto and @thegamingbat) would like to work on this. Happy to discuss the design further before coding.
### Affected Area
Workflow Engine (Amber), Deployment / Infrastructure
Contributor guide
Assessment
This issue has not been assessed yet.