[Notifications] OpenTelemetry instrumentation and the locked metric contract
- Dominant language
- Go
- Stars
- 55
- Forks
- 16
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 87
Description
Wire OTel into `fleet-api` and freeze the metric names and labels. The metric contract is locked before UI work because renames mean rewriting every shipped rule.
Checklist:
- [ ] New package `server/internal/infrastructure/metrics` owning the `MeterProvider`, OTLP gRPC exporter, and resource attributes. Endpoint from `OTEL_EXPORTER_OTLP_ENDPOINT` (default `http://otel-collector:4317`). Lifecycle wired in `server/cmd/fleetd`.
- [ ] `metrics/contract.go` declares Go constants for every metric name and label key from the design's contract table: `fleet_device_online`, `fleet_device_hashrate_terahash`, `fleet_device_hashrate_expected_terahash`, `fleet_device_temperature_max_celsius`, `fleet_device_temperature_avg_celsius`, `fleet_device_pool_connected`, `fleet_command_total`, `fleet_telemetry_poll_total`. Documented in `docs/notifications-metrics.md`.
- [ ] Subscriber on `server/internal/domain/telemetry/broadcaster.go` maps telemetry events to instrument observations across Antminer, Proto, asicrs, and virtual plugins. Per-board and per-chip detail exposed only as pre-aggregated `_max` and `_avg`.
- [ ] Counter wiring: `fleet_command_total{kind,result}` in `server/internal/domain/command`; `fleet_telemetry_poll_total{result}` in the telemetry scheduler. `result` enum constrained to `success|failure`.
- [ ] When the telemetry pipeline marks a device unreachable, emit explicit `fleet_device_online=0` rather than dropping the series. Default rules also include an `absent_over_time(fleet_device_online[10m])` guard.
- [ ] Linter / unit test asserts no instrument is created outside the `fleet_` namespace and no label outside the allowlist is used.
- [ ] Cardinality budget test: 1,000 devices × 50 groups × 5 orgs stays under the documented budget.
- [ ] In-process OTLP receiver test verifies metrics flow end-to-end.
Contributor guide
Assessment
This issue has not been assessed yet.