cockroachdb / cockroachdb/cockroach

allocator: reduce memory churn

Open
#105,349 0 comments 0 reactions 0 assignees View on GitHub
A-kv-distribution C-enhancement T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

The amount of memory the allocator uses scales linearly with the number of stores being considered. In a cluster with many stores, the memory churn can lead to GC pressure which results in increased time spent collecting and potentially scheduler contention.

See the screenshot taken below of the recent allocations (5s) on a node in a 60 node cluster:

![image](https://github.com/cockroachdb/cockroach/assets/39606633/6908a6f0-6c5c-4f6a-824c-a10ec16fc90f)

**Describe the solution you'd like**
Wrapping `rankedCandidateListForRebalancing` and `StoreList` in a sync pool would get a lot of mileage.

**Extra Context**

Note the multi-metric allocator, allocator2 already does most of this. An approach which can be backported to 23.1 and potentially 22.2 would be ideal.

Jira issue: CRDB-28996

Epic CRDB-41111

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.