proxy: tune latency histogram buckets

Nobody has claimed this yet.

### What problem are you trying to solve?

The latency percentiles calculated by Linkerd Viz are highly inaccurate for my workload. I have deployed a demo service that produces a HTTP response after 5 seconds. In production, this service can take either 5000ms or 5001ms to respond. Because the closest [buckets](https://github.com/linkerd/linkerd2-proxy/blob/d315b9f6afb23eb3d0cf58577710989d1bd69944/lib/metrics/src/latency.rs#L7-L35) are 5000 and 10000, the p95 latency is calculated as 9750, however, not a single request takes longer than 5001.

I'm aware that this is an exact duplicate of [another issue](https://github.com/linkerd/linkerd2/issues/3202), but hopefully this adds more weight to the issue. It's something I'm facing on production on a daily basis.

### How should the problem be solved?

Buckets should be sized according to expected latency. There are a lot of assumptions around latencies with developers saying things like ["60 seconds ought to be enough"](https://github.com/linkerd/linkerd2/pull/1218#discussion_r198655964), or "Prometheus maxes out at 10 seconds". Personally I think this is dangerous - Observability is a main pillar of linkerd and is arguable most interesting in context of services that have become slow. Using Prometheus inherently introduces limitations, so configuration should be possible by the end user.

For example, bucket configuration can be added to Service Profiles (ideally on a per route basis). It could be either a list of buckets, or, preferably, some parameters, for example:
- equally sized buckets from 0 to 5 seconds with a maximum 5% measurement error
- non-equally sized buckets from 0 to 5 seconds with a measurement error ranging from 1 to 10% with an expected latency of 800ms. This would result in smaller buckets around the 800ms mark.

[Further reading on the subject](https://thenewstack.io/how-to-correctly-frame-and-calculate-latency-slos/)

### Any alternatives you've considered?

I could enable HTTP access logs in Linkerd and try to extract the histograms out of those. It is significantly more effort as a completely separate log pipeline has to be built and is limited to HTTP requests. gRPC services won't be instrumented.

Implementing a cluster wide logging infrastructure for k8s for only this feature is a massive undertaking and I'd probably just add another reverse proxy like nginx in front of every application - adding yet another reverse proxy :(

---

A more convenient solution for the end user which requires significantly more engineering effort:
The proxy could implement [circllhist](https://arxiv.org/pdf/2001.06561.pdf) which is a merge-able data-structure for quantiles that does not need to be configured and uses very wide boundaries:

> The largest representable number of the circllhist is 99 · 10127. This number is larger than the age of the universe
measured in nano-seconds (13.8 billion years) The smallest representable positive number is 10 · 10−128. This number
is smaller than the Plank time measured in years (5.39 · 10−44 s). We have found this value range to be sufficient for all
practical purposes.

This data then needs to be scraped from the linkerd proxy and merged in a central place.

### How would users interact with this feature?

_No response_

### Would you like to work on this feature?

_No response_

Start with lib/metrics/src/latency.rs and inspect the existing bucket definitions and how the proxy exposes them to Prometheus. Review the linked duplicate issue and the proposed Service Profile configuration or circllhist alternatives before choosing a design. Done means latency percentiles remain accurate for workloads such as 5000–5001ms and the chosen configuration is usable by end users.

Written by the indexing model from the issue text.