cockroachdb / cockroachdb/cockroach
workload/histogram: migrate the workload histogram framework off hdrhistogram to goodhistogram
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Summary**
The workload latency framework (`pkg/workload/histogram`) records, merges, and serializes latencies via `github.com/codahale/hdrhistogram`, and `hdrhistogram` types leak through its public API (`Tick.Hist`, `Tick.Cumulative`, the `Exporter` interface, `SnapshotTick.Hist`). It is the largest remaining consumer of the dependency; together with `bulkpb` and the `pkg/util/metric` HDR model it blocks dropping `hdrhistogram` from `go.mod`.
**Key constraint**
`stats.json` persists the raw HDR snapshot (`{lowest, highest, sigFigs, Counts}`). HDR's bucket geometry (linear-within-exponential) differs from goodhistogram's (pure exponential), so goodhistogram cannot reproduce that byte format. However, no consumer needs the raw geometry — they use only quantiles, merge, and count: in-repo consumers (`tpcc`, `kvbench`, `kv_rangefeed`, `tpcc/result.go`) `Import` then call `ValueAtQuantile`/`Merge`, and the newer OpenMetrics export emits only p50/p95/p99/p100 summaries. The raw `Counts` array is just a transport for recomputing quantiles/merging later.
**Sequencing**
This is effectively gated on the in-flight HDR-JSON → OpenMetrics export migration: once roachperf and the perf roachtests consume OpenMetrics summaries rather than the raw HDR `stats.json`, swapping the recorder to goodhistogram (`Snapshot.Merge` for tick/cumulative aggregation) is near-mechanical. Involves cross-team (test-eng/perf) and cross-repo (roachperf, historical artifacts) coordination, so likely the last of the three HDR consumers to land.
**Tasks**
- [ ] Complete the OpenMetrics export cutover so the raw HDR `stats.json` snapshot is no longer the consumed format (coordinate roachperf + historical artifacts).
- [ ] Replace the `hdrhistogram` types in `pkg/workload/histogram` (`NamedHistogram`/`Registry`/`Tick`) and the exporters with goodhistogram.
- [ ] Update consumers: `tpcc/result.go`, `syncbench`, and the perf roachtests.
- [ ] Remove the `codahale/hdrhistogram` import once this, `bulkpb`, and the metric model are all done — then drop it from `go.mod`.
Epic CRDB-63921
Jira issue: CRDB-64881
Contributor guide
Research direction
Start with pkg/workload/histogram and its NamedHistogram, Registry, Tick, SnapshotTick, and exporter APIs, then inspect tpcc/result.go, syncbench, and the perf roachtests. Confirm the OpenMetrics cutover and roachperf coordination first; done means consumers no longer depend on raw HDR stats.json, goodhistogram replaces the exposed HDR types, and the HDR dependency can be removed after the other consumers are migrated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100