Aiven-Open / Aiven-Open/karapace
Expose Schema Registry health status as Prometheus metrics
- Dominant language
- Python
- Stars
- 634
- Forks
- 110
- Avg merge
- 4d 7h
- Merged PRs (30d)
- 4
Description
# What is currently missing?
Karapace exposes operational state through `/_health`, but most of those values are unavailable as Prometheus metrics on `/metrics`.
Prometheus users therefore cannot directly alert on or graph Schema Registry readiness, reader progress/lag, coordinator state, or primary-election state without separately polling and transforming the JSON health endpoint.
# How could this be improved?
Expose the bounded, operationally useful health values as Prometheus-native metrics from a shared health-state model used by both `/_health` and `/metrics`.
Proposed metrics:
* `karapace_health`
* `karapace_schema_registry_ready`
* `karapace_schema_registry_startup_duration_seconds`
* `karapace_schema_registry_reader_current_offset`
* `karapace_schema_registry_reader_highest_offset`
* `karapace_schema_registry_reader_lag`
* `karapace_schema_registry_primary`
* `karapace_schema_registry_primary_eligible`
* `karapace_schema_registry_coordinator_running`
* `karapace_schema_registry_coordinator_generation`
* `karapace_build_info{version="..."}`
* `karapace_health_check_timestamp_seconds`
The primary URL should not become a metric label because it may expose topology and introduces unnecessary label cardinality.
Most fields are inexpensive in-memory values. Overall health currently requires the asynchronous Kafka check performed by `KafkaSchemaReader.is_healthy()`. The synchronous Prometheus collector should not block on Kafka or bridge synchronously into the event loop. The implementation should instead share health state between the endpoint and metrics and make cached-health freshness observable.
## Acceptance criteria
* `/metrics` exposes health, readiness, reader, and coordinator metrics.
* Values use the same underlying state and semantics as `/_health`.
* Prometheus collection does not perform blocking Kafka I/O.
* Cached health freshness is observable.
* Primary, replica, and unknown states remain distinguishable.
* No unbounded-cardinality labels are introduced.
* Existing configured metric tags are applied where appropriate.
* Unit tests cover health states, primary states, offsets, and lag.
* An integration test verifies the new series on `/metrics`.
* Observability documentation lists the metrics and example alerts.
# Is this a feature you would work on yourself?
* [x] I plan to open a pull request for this feature
Contributor guide
Assessment
This issue has not been assessed yet.