cockroachdb / cockroachdb/cockroach

changefeedccl: impact of many changefeeds on cluster

Open
#101,503 1 comment 0 reactions 0 assignees View on GitHub
A-cdc C-enhancement T-cdc
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

As a follow up to the cdc pre-mortem: https://docs.google.com/document/d/1X2D64G_Jdb9S4MxrBYPH_fwKAP3dkUrhc7yUgZZNZAY/edit#

We're worried about CDC performance when running 100, 1k, 10k, …. or more changefeeds in a cluster. What is a reasonable impact? Will the job system work? How about the console? Will debug.zip capturing work? What about monitoring? Is this a supported use case? If not, what’s the limit? Having frontier processors running (many) on one node is going to cause problems we expect.

We should investigate and address the case of 10xing the number of changefeeds on a cluster, from 50->500.

Jira issue: CRDB-26976

Epic CRDB-9398

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.