cockroachdb / cockroachdb/cockroach

Different patterns in the avro schema registry for continuous and scheduled changefeed creation

Open
#148,518 2 comments 0 reactions 0 assignees View on GitHub
A-cdc C-bug T-cdc
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Please describe the issue you observed, and any steps we can take to reproduce it:

When creating a continuous vs scheduled changefeed, the resulting avro schema registry changes from `crdb___-values` to `..-values`

The steps were as followings:

1. Create connection for continuous change feed:
"CREATE EXTERNAL CONNECTION <...> AS " +
"'kafka://<...>" +
"sasl_enabled=true&sasl_mechanism=SCRAM-SHA-512&" +
"sasl_user=<...>&sasl_password=<...>" +
"tls_enabled=true&ca_cert=<>&topic_prefix=crdb.v1'"

2. Create Change feed:
"CREATE CHANGEFEED FOR TABLE <...>.<...> INTO 'external://<...>' WITH OPTIONS (" +
"confluent_schema_registry = '<...>', " +
"format = 'avro', updated, full_table_name)"

3. Create connection for Scheduled change feed snapshot:
"CREATE EXTERNAL CONNECTION <...> AS " +
"'kafka://<...>" +
"sasl_enabled=true&sasl_mechanism=SCRAM-SHA-512&" +
"sasl_user=<...>&sasl_password=<...>" +
"tls_enabled=true&ca_cert=<>&topic_prefix=crdb.snapshot.v1.'"

4. Create Scheduled Change feed snapshot:
"CREATE SCHEDULE IF NOT EXISTS <...> FOR CHANGEFEED " +
"INTO 'external://<...>' " +
"WITH OPTIONS (confluent_schema_registry = 'https://<...>', format=avro, envelope=wrapped, full_table_name) " +
"AS SELECT *, statement_timestamp() AS sync_ts FROM <...>.<...> " +
"RECURRING '<...>' WITH SCHEDULE OPTIONS (on_previous_running=skip, on_execution_failure=retry, first_run=now)"

**Environment:**
- CockroachDB version [e.g. 2.0.x]
- Server OS: [e.g. Linux/Distrib]
- Client app [e.g. `cockroach sql`, JDBC, ...]

**Additional context**
None at the moment

Jira issue: CRDB-51659

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.