cockroachdb / cockroachdb/cockroach

changefeedccl: Send changefeed configuration to downstream receivers

Open
#122,185 2 comments 0 reactions 0 assignees View on GitHub
A-cdc C-enhancement T-cdc
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Is your feature request related to a problem? Please describe.**

In the current state, a changefeed receiver can not know, for example, all tables that may be part of a changefeed. This makes it awkward for a receiver to act upon groups of tables if not all tables that are part of the changefeed have had any messages to emit.

Consider this scenario: It would be nice to be able to use resolved timestamps as a measure of replication progress made for any table that is part of a changefeed. If, for example, a customer starts a changefeed that contains an empty table or uses a (backup/restore + changefeed cursor) over a reference table which rarely changes, the receiver cannot report a complete picture of the tables in the feed.

**Describe the solution you'd like**

MVP: Add additional metadata to the resolved-timestamp messages to include all table names that the timestamp applies to.

Ideal: Add a new message type that describes the changefeed tables, their schemas, and other information that would also be part of a solution for #112880. This changefeed-schema message would be emitted upon changefeed startup or in response to underlying table or query-projection changes.

**Describe alternatives you've considered**

Currently, an operator is required to synchronize the list of tables in the source changefeed with the changfeed receiver, to know the entire set of tables that may appear within the feed. This creates an opportunity for configuration drift between multiple systems.

Jira issue: CRDB-37749

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.