cockroachdb / cockroachdb/cockroach

workload/histogram: migrate the workload histogram framework off hdrhistogram to goodhistogram

Open
#171,746 0 comments 0 reactions 0 assignees View on GitHub
A-observability-inf C-enhancement O-agent T-observability
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Summary**

The workload latency framework (`pkg/workload/histogram`) records, merges, and serializes latencies via `github.com/codahale/hdrhistogram`, and `hdrhistogram` types leak through its public API (`Tick.Hist`, `Tick.Cumulative`, the `Exporter` interface, `SnapshotTick.Hist`). It is the largest remaining consumer of the dependency; together with `bulkpb` and the `pkg/util/metric` HDR model it blocks dropping `hdrhistogram` from `go.mod`.

**Key constraint**

`stats.json` persists the raw HDR snapshot (`{lowest, highest, sigFigs, Counts}`). HDR's bucket geometry (linear-within-exponential) differs from goodhistogram's (pure exponential), so goodhistogram cannot reproduce that byte format. However, no consumer needs the raw geometry — they use only quantiles, merge, and count: in-repo consumers (`tpcc`, `kvbench`, `kv_rangefeed`, `tpcc/result.go`) `Import` then call `ValueAtQuantile`/`Merge`, and the newer OpenMetrics export emits only p50/p95/p99/p100 summaries. The raw `Counts` array is just a transport for recomputing quantiles/merging later.

**Sequencing**

This is effectively gated on the in-flight HDR-JSON → OpenMetrics export migration: once roachperf and the perf roachtests consume OpenMetrics summaries rather than the raw HDR `stats.json`, swapping the recorder to goodhistogram (`Snapshot.Merge` for tick/cumulative aggregation) is near-mechanical. Involves cross-team (test-eng/perf) and cross-repo (roachperf, historical artifacts) coordination, so likely the last of the three HDR consumers to land.

**Tasks**
- [ ] Complete the OpenMetrics export cutover so the raw HDR `stats.json` snapshot is no longer the consumed format (coordinate roachperf + historical artifacts).
- [ ] Replace the `hdrhistogram` types in `pkg/workload/histogram` (`NamedHistogram`/`Registry`/`Tick`) and the exporters with goodhistogram.
- [ ] Update consumers: `tpcc/result.go`, `syncbench`, and the perf roachtests.
- [ ] Remove the `codahale/hdrhistogram` import once this, `bulkpb`, and the metric model are all done — then drop it from `go.mod`.

Epic CRDB-63921

Jira issue: CRDB-64881

Contributor guide

Open the contributing guide

Research direction

Start with pkg/workload/histogram and its NamedHistogram, Registry, Tick, SnapshotTick, and exporter APIs, then inspect tpcc/result.go, syncbench, and the perf roachtests. Confirm the OpenMetrics cutover and roachperf coordination first; done means consumers no longer depend on raw HDR stats.json, goodhistogram replaces the exposed HDR types, and the HDR dependency can be removed after the other consumers are migrated.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.