cockroachdb / cockroachdb/cockroach

changefeedccl: decouple changefeed job info and checkpoint data

Open
#125,024 1 comment 0 reactions 0 assignees View on GitHub
A-cdc C-enhancement T-cdc
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Today, changefeed frontiers update their legacy_progress row in the job info table whenever checkpointing occurs. This creates unnecessary contention on the job info table, since the legacy_progress is queried for tasks like SHOW JOBS, but checkpoint data is not necessary for these queries. We could reduce contention by decoupling the machine-readable checkpoint data and moving it to its own row (or rows) in the job info table. We would still need to periodically update information like the high water mark in the legacy_progress entry, but could do this at a lower frequency than checkpointing.

Jira issue: CRDB-39197

Epic CRDB-41789

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.