cockroachdb / cockroachdb/cockroach

roachtest: introduce `test_run_info` metric for roachtests

Open
#163,568 1 comment 0 reactions 0 assignees View on GitHub
A-testeng-perf C-enhancement T-testeng
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

We currently rely on high-cardinality metrics (e.g. `sys_uptime`) to discover test metadata such as `test_run_id`. Grafana queries like `/api/v1/label/test_run_id/values?match=...` force Prometheus to scan TSDB blocks in the selected time range to compute label values, since these labels are not fully indexed in memory. This has resulted in very slow queries (~40s observed). VictoriaMetrics exhibits similar behavior.

We need to support rich run-level metadata for competitive benchmarking, roachperf, and roachtest, to power dashboard selectors from those labels, and avoid attaching all metadata labels to every detailed metric (to prevent cardinality explosion).

**Proposal**

Introduce a new metric, `test_run_info`, emitting exactly one sample per test run. This metric will contain run-level labels such as `test_run_id` (short, unique ID), `machine_type`, `store_count`, `disk_type`, `disk_size`, and other relevant metadata. Detailed metrics should include only `test_run_id` and rely on `test_run_info` for all other run metadata.

This separates run metadata from detailed metrics, reduces label union size, improves selector query performance, and scales better over time.

Jira issue: CRDB-60296

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.