cockroachdb / cockroachdb/cockroach

kv,rpc: adopt static labels

Open
#150,493 3 comments 0 reactions 0 assignees View on GitHub
A-kv-observability C-enhancement E-easy E-starter T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

# Metric Refactoring for Static Labels

This issue tracks the migration of existing metrics to use the new static labels feature[^1]. This will help reduce metric cardinality and improve queryability.

[^1]: https://docs.google.com/document/d/1fn5KX0uPUCw1uW4KkCvmP9-svga0F6XSvhnqdg24MpE/edit?tab=t.0#heading=h.9ndo02nsmu

I'm particularly bullish on the first three because it'll be so much easier (on DD) to add one metric and to get to see all the flavors of requests.

Some of the others might be useful as well. I prepared the below using Cursor/Gemini2.5-pro by pointing it at metrics.html and giving it a .txt version of the static labels feature doc.

## DistSender

- [ ] `distsender.rpc.sent`: `method=(addsstable,adminchangereplicas,adminmerge,adminrelocaterange,adminscatter,adminsplit,admintransferlease,adminunsplit,adminverifyprotectedtimestamp,barrier,checkconsistency,clearrange,computechecksum,conditionalput,delete,deleterange,endtxn,excise,export,flushlocktable,gc,get,heartbeattxn,increment,initput,isspanempty,leaseinfo,linkexternalsstable,merge,migrate,probe,pushtxn,put,queryintent,querylocks,queryresolvedtimestamp,querytxn,rangestats,recomputestats,recovertxn,refresh,refreshrange,requestlease,resolveintent,resolveintentrange,reversescan,revertrange,scan,subsume,transferlease,truncatelog,writebatch)`
- [ ] `distsender.rpc.errors`: `error=(ambiguousresulterrtype,batchtimestampbeforegcerrtype,communicationerrtype,conditionfailederrtype,errordetailtype(0),errordetailtype(15),errordetailtype(19),errordetailtype(20),errordetailtype(21),errordetailtype(23),errordetailtype(24),errordetailtype(29),errordetailtype(30),errordetailtype(33),indeterminatecommiterrtype,integeroverflowerrtype,intentmissingerrtype,internalerrtype,invalidleaseerrtype,leaserejectederrtype,lockconflicterrtype,mergeinprogresserrtype,mintimestampboundunsatisfiableerrtype,mvcchistorymutationerrtype,nodeunavailableerrtype,notleaseholdererrtype,oprequirestxnerrtype,optimisticevalconflictserrtype,proxyfailederrtype,raftgroupdeletederrtype,rangefeedretryerrtype,rangekeymismatcherrtype,rangenotfounderrtype,readwithinuncertaintyintervalerrtype,refreshfailederrtype,replicacorruptionerrtype,replicatooolderrtype,replicaunavailableerrtype,storenotfounderrtype,transactionabortederrtype,transactionpusherrtype,transactionretryerrtype,transactionretrywithprotorefresherrtype,transactionstatuserrtype,txnalreadyencounterederrtype,unsupportedrequesterrtype,writeintenterrtype,writetooolderrtype)`

## RPC

- [ ] `rpc.requests_received`: `method=(addsstable,adminchangereplicas,adminmerge,adminrelocaterange,adminscatter,adminsplit,admintransferlease,adminunsplit,adminverifyprotectedtimestamp,barrier,checkconsistency,clearrange,computechecksum,conditionalput,delete,deleterange,endtxn,excise,export,flushlocktable,gc,get,heartbeattxn,increment,initput,isspanempty,leaseinfo,linkexternalsstable,merge,migrate,probe,pushtxn,put,queryintent,querylocks,queryresolvedtimestamp,querytxn,rangestats,recomputestats,recovertxn,refresh,refreshrange,requestlease,resolveintent,resolveintentrange,reversescan,revertrange,scan,subsume,transferlease,truncatelog,writebatch)`

## Raft

- [ ] `raft.rcvd.messages`: `type=(app,appresp,defortifyleader,dropped,fortifyleader,fortifyleaderresp,heartbeat,heartbeatresp,prevote,prevoteresp,prop,snap,timeoutnow,transferleader,vote,voteresp)`

## Admission Control

- [ ] `admission.requests`: `status=(admitted,errored,requested)` `workload=(elastic-cpu,elastic-stores,kv,kv-stores,sql-kv-response,sql-sql-response)` `priority=(bulk-normal-pri,normal-pri,bulk-low-pri,high-pri,locking-normal-pri,user-high-pri,none)`
- [ ] `admission.wait_duration`: `workload=(elastic-cpu,elastic-stores,kv,kv-stores,snapshot_ingest,sql-kv-response,sql-sql-response)` `priority=(bulk-normal-pri,normal-pri,bulk-low-pri,high-pri,locking-normal-pri,user-high-pri,none)`
- [ ] `admission.wait_queue_length`: `workload=(elastic-cpu,elastic-stores,kv,kv-stores,sql-kv-response,sql-sql-response)` `priority=(bulk-normal-pri,normal-pri,bulk-low-pri,high-pri,locking-normal-pri,user-high-pri,none)`

## Allocator

- [ ] `kv.allocator.load_based_lease_transfers`: `reason=(cannot_find_better_candidate,delta_not_significant,existing_not_overfull,follow_the_workload,missing_stats_for_existing_stores,should_transfer)`
- [ ] `kv.allocator.load_based_replica_rebalancing`: `reason=(cannot_find_better_candidate,delta_not_significant,existing_not_overfull,missing_stats_for_existing_store,should_transfer)`

## KV Prober

- [ ] `kv.prober.attempts`: `operation=(planning,read,write)`
- [ ] `kv.prober.failures`: `operation=(planning,read,write)`
- [ ] `kv.prober.latency`: `operation=(read,write)`

## Transactions

- [ ] `txnrecovery.successes`: `outcome=(aborted,committed,pending)`

## Snapshots

- [ ] `range.snapshots.applied`: `type=(initial,non-voter,voter)`
- [ ] `range.snapshots.sent.bytes`: `reason=(total,cross-region,cross-zone,rebalancing,recovery,unknown,upreplication)`
- [ ] `range.snapshots.rcvd.bytes`: `reason=(total,cross-region,cross-zone,rebalancing,recovery,unknown,upreplication)`

## Queues

- [ ] `queue.replicate.removals`: `reason=(dead,decommissioning,learner,rebalance)` `type=(any,nonvoter,voter)`

Jira issue: CRDB-52738

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.