cockroachdb / cockroachdb/cockroach

cdc: revisit pts related cluster settings

Open
#129,529 2 comments 0 reactions 0 assignees View on GitHub
A-cdc C-enhancement T-cdc
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Is your feature request related to a problem? Please describe.**

Previously, we always update PTS record if the last time we updated pts has passed by `changefeed.protect_timestamp_interval` and if a checkpoint happens. However, the `lastProtectedTimestampUpdate` is saved on frontiers, so this info is lost whenever chanegfeed restarts. This could lead to lots of PTS updates if changefeeds keep restarting.

To solve this problem, we introduced another cluster setting to allow PTS updates only if it is lagging behind high watermark by changefeed.protect_timestamp.lag in this PR https://github.com/cockroachdb/cockroach/pull/129442.

We now have two cluster settings to guard how frequent PTS updates happen. It makes things harder to rationalize. We should discuss and see if we should just keep one.

Jira issue: CRDB-41575

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.