Support native histograms
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 11.5k
- Forks
- 1.4k
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 53
Description
### What problem are you trying to solve?
With each linkerd-proxy sidecar exporting histogram metrics, they're quickly becoming the dominant metrics in our Prometheus instances. There are also other benefits. Switching to native histograms (stable since Prometheus 3.8.0) would:
* decrease storage requirements,
* speed up various operations in Prometheus, including queries (due to huge cardinality drop),
* improve outlier reporting (native histograms define precision, not buckets).
See detailed comparison in [Prometheus docs](https://prometheus.io/docs/practices/histograms/).
### How should the problem be solved?
* Add a switch in Viz config that would change between native and classic histogram queries.
* Add native histograms to proxies and other components. There are at least two approaches possible.
* Always export both classic and native histograms (Prometheus has options to only scrape one of them).
* Add a config option (annotation in case of proxy?) that would allow selecting one of them.
Important note: native histograms only work with the protobuf encoding.
### Any alternatives you've considered?
It's possible to convert classic histograms losslessly to NHCB (Native Histograms with Custom Buckets) by tuning Prometheus scrape config. In this approach, no changes to exported metrics are required.
**Viz queries still need to be updated.**
### How would users interact with this feature?
Most likely a single boolean in the main helm chart and another one per extension. In case of Viz, it might either try to query both classic and native histograms or have a config option for that.
### Would you like to work on this feature?
maybe
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing Viz config and queries, the main Helm chart and per-extension settings, and histogram export in linkerd-proxy and other components. Check the Prometheus protobuf-encoding constraint before choosing between dual export and a selection option. Done means native-histogram support is configurable, proxy and component exports are covered, and Viz queries handle the selected histogram type.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, helm, kubernetes, prometheus, rust
- Domain
- cloud, distributed-systems, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100