cockroachdb / cockroachdb/cockroach
changefeedccl: decouple changefeed job info and checkpoint data
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
Today, changefeed frontiers update their legacy_progress row in the job info table whenever checkpointing occurs. This creates unnecessary contention on the job info table, since the legacy_progress is queried for tasks like SHOW JOBS, but checkpoint data is not necessary for these queries. We could reduce contention by decoupling the machine-readable checkpoint data and moving it to its own row (or rows) in the job info table. We would still need to periodically update information like the high water mark in the legacy_progress entry, but could do this at a lower frequency than checkpointing.
Jira issue: CRDB-39197
Epic CRDB-41789
Contributor guide
Assessment
This issue has not been assessed yet.