cockroachdb / cockroachdb/cockroach

kvserver: use cached store capacity in more calls

Open
#94,929 2 comments 0 reactions 0 assignees View on GitHub
A-kv-distribution C-enhancement T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Is your feature request related to a problem? Please describe.**

`store.Capacity()` is expensive as it iterates over every replica. It is called periodically on the same interval by different tickers, however we don't use the cached value - meaning that we wastefully recompute it from scratch despite it not being very stale.

This wastes CPU resources, especially in a high replica count cluster.

We calculate the store capacity periodically from these places:

- [node.computeMetricsPeriodically()](https://github.com/cockroachdb/cockroach/blob/6c619e285f78a15f04a9ca75faf746ae93530f6b/pkg/server/node.go#L770-L770)
- 14.83% cpu
- [every 10s](https://github.com/cockroachdb/cockroach/blob/6c619e285f78a15f04a9ca75faf746ae93530f6b/pkg/base/config.go#L97-L97)
- [node.gossipStores()](https://github.com/cockroachdb/cockroach/blob/6c619e285f78a15f04a9ca75faf746ae93530f6b/pkg/server/node.go#L736-L736)
- 9.74% cpu
- [every 10s](https://github.com/cockroachdb/cockroach/blob/6c619e285f78a15f04a9ca75faf746ae93530f6b/pkg/base/config.go#L97-L97)
- [node.writeNodeStatus()](https://github.com/cockroachdb/cockroach/blob/6c619e285f78a15f04a9ca75faf746ae93530f6b/pkg/server/node.go#L992-L992)
- 10.10% cpu
- [every 10s](https://github.com/cockroachdb/cockroach/blob/6c619e285f78a15f04a9ca75faf746ae93530f6b/pkg/base/config.go#L97-L97)

**Describe the solution you'd like**

Use `Capacity(true /* useCached */)` more regularly, rather than have each ticking component which is calling at the same interval calculate from scratch. Ideally, this useCached would switch to a staleness interval the caller is willing to accept the cached value from before recalculating.

Jira issue: CRDB-23231

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.