cockroachdb / cockroachdb/cockroach
changefeedccl: leverage multiple job info rows for the lag checkpoint instead of truncating it
Open
A-cdc
C-enhancement
T-cdc
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
instead of squeezing the lag checkpoint into one `legacy_progress` job_info row, truncating it in the process, we can spread it across multiple rows with different `info_key`s. This would remove the need for truncation.
We can either use [this helper](https://github.com/asg0451/cockroach/blob/c3a4d77100ac6fcc3b9144bf6efc99e2829b007c/pkg/jobs/job_info_utils.go#L29) or do something more intelligent with knowledge of the frontier's structure (e.g. partition by table or span or something).
Jira issue: CRDB-50535
Contributor guide
Assessment
This issue has not been assessed yet.