cockroachdb / cockroachdb/cockroach

LDR: Create Replicated Table Should Pre-split Ranges

Open
#144,879 0 comments 0 reactions 0 assignees View on GitHub
C-enhancement
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

LDR is implemented as a distsql flow that places a processor on each node in the cluster. As a result the concurrency is proportional to the number of nodes in the cluster. This allows LDR to horizontally scale with cluster size, but can lead to instability when a large cluster is replicating into a small table.

This is particularly problematic when starting LDR for the first time. When a table is first created, it maps to a single range. If LDR is started on a 100 node cluster, every node in the cluster is attempting to replicate data into the lone range. This will overload nodes serving that range until the range has an opportunity to split and scatter across the cluster.

This is made worse by the fact that [LDR does not slew goroutine creation when it receives a replication event batch](https://github.com/cockroachdb/cockroach/issues/144877). In a 100 node cluster, this leads to 10K goroutines attempting to write to a single range all at once.

The [CREATE LOGICALLY REPLICATED TABLE](https://www.cockroachlabs.com/docs/v25.1/create-logically-replicated) command should be augmented so that the ranges in the destination cluster are pre-split.

Jira issue: CRDB-49588

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.