cockroachdb / cockroachdb/cockroach

kvserver: re-evaluate default merge queue interval cluster setting

Open
#168,575 1 comment 0 reactions 0 assignees View on GitHub
branch-master C-bug O-support P-3 T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Describe the problem**

Because of the expense of merges, the default merge queue interval is 5 seconds (e668bfe8d21556d8970456e146e4000768faea03). As a result, a given store only processes about 12 replicas per minute. As a result, once the number of replica's exceeds the queue max, the merge queue is perpetually full and it can take a considerable amount of time before a given range is actually processed by the merge queue.

In clusters that have a high rate of producing many small ranges, the cap on the merge queue can result in an ever growing range counts. This is sometimes caused by workloads that do a large number of scheme changes or workloads that consistently truncate tables.

Even in clusters without those workloads, we occasionally find that lower the merge queue interval for a brief period of time results in a substantial range count reduction because of small ranges that scanner was simply unable to ever add to the queue because it was full.

Jira issue: CRDB-63016

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.