cockroachdb / cockroachdb/cockroach

kv/bulk: migrate IngestionPerformanceStats histograms off hdrhistogram to goodhistogram

Open
#171,745 0 comments 0 reactions 0 assignees View on GitHub
A-observability-inf C-enhancement O-agent T-observability
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Summary**

`IngestionPerformanceStats` (`pkg/kv/bulk/bulkpb`) embeds two HDR histograms — `BatchWaitHist` and `SstSizeHist` — in the `HistogramData` proto, recorded into tracing spans and combined across nodes into a job's execution-detail files. This is the bulk-ingestion use of `github.com/codahale/hdrhistogram`; migrating it to `goodhistogram` is part of the goodhistogram epic. (The workload framework and the `pkg/util/metric` HDR model are separate consumers; the `go.mod` dependency only drops once all three are migrated.)

**Scope**

Replace the codahale/hdrhistogram usage in `bulkpb` with goodhistogram, behind a cluster-version gate. Affects only the execution-detail diagnostics of ingestion jobs (RESTORE / IMPORT / index backfills / PCR / LDR).

**Compatibility note**

`HistogramData` crosses nodes inside `TracingAggregatorEvents` (DistSQL producer metadata) and is rendered on demand via `crdb_internal.request_job_execution_details`. A format change needs a version gate so new-format data isn't sent to nodes that can't read it. Risk is low and debug-only — bounded to the mixed-version upgrade window, affecting one field of an on-demand debug artifact. Cross-node `Combine` records scalar values rather than merging the remote histogram's bucket array, so there is no cross-version bucket-merge hazard.

**Tasks**
- [ ] Add goodhistogram-backed histogram field(s) to the bulkpb proto (deprecate the HDR fields).
- [ ] Gate population of the new format on cluster version.
- [ ] Port `getCombinedHist` and the `String`/`Render` paths to goodhistogram.
- [ ] Remove the `codahale/hdrhistogram` import from `pkg/kv/bulk/bulkpb` once the gate permits.

Epic CRDB-63921

Jira issue: CRDB-64880

Contributor guide

Open the contributing guide

Research direction

Start in pkg/kv/bulk/bulkpb by tracing HistogramData, IngestionPerformanceStats, getCombinedHist, and the String/Render paths, then inspect how the data moves through TracingAggregatorEvents. Review the existing cluster-version gating patterns before changing the proto fields. Done means the goodhistogram format is gated, rendered and combined correctly, and the bulkpb HDR import is removed when permitted.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, databases, distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.