cockroachdb / cockroachdb/cockroach

changefeedccl: leverage multiple job info rows for the lag checkpoint instead of truncating it

Open
#146,464 1 comment 0 reactions 0 assignees View on GitHub
A-cdc C-enhancement T-cdc
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

instead of squeezing the lag checkpoint into one `legacy_progress` job_info row, truncating it in the process, we can spread it across multiple rows with different `info_key`s. This would remove the need for truncation.

We can either use [this helper](https://github.com/asg0451/cockroach/blob/c3a4d77100ac6fcc3b9144bf6efc99e2829b007c/pkg/jobs/job_info_utils.go#L29) or do something more intelligent with knowledge of the frontier's structure (e.g. partition by table or span or something).

Jira issue: CRDB-50535

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.