metric: improved testing for histogram latency buckets

**Is your feature request related to a problem? Please describe.**
Recently we had an incident where the lack of resolution in the histogram quantiles time series caused us to report inaccurate latency data.

Related Issue: https://github.com/cockroachdb/cockroach/issues/95833
Postmortem: https://cockroachlabs.atlassian.net/wiki/spaces/OI/pages/2884632706/2023-01-27+Postmortem+on+Low+Resolution+Metric+Histograms+Impact+Latency+Observability

**Describe the solution you'd like**
Some sort of test that looks at latencies reported by crdb histograms and compares it to the actual latency when running a workload to ensure it is under an acceptable threshold (TBD what that threshold is). This will help catch inappropriately designed histogram buckets for a given metric since they are statically defined in https://github.com/cockroachdb/cockroach/blob/master/pkg/util/metric/histogram_buckets.go.

During the postmortem, it was suggested to try and incorporate this as part of roachperf or roachtest as they are run regularly and would have caught this problem earlier.

Jira issue: CRDB-24540

Epic CRDB-20790

Start by reading pkg/util/metric/histogram_buckets.go and the roachperf or roachtest entry points mentioned in the issue. Define the acceptable latency-error threshold and identify a workload that compares reported histogram quantiles with actual latency. Done means a regularly run test detects insufficient histogram resolution.

Written by the indexing model from the issue text.