cockroachdb / cockroachdb/cockroach

externalconnection,changefeed: consider UX of kafka query parameters for external connections

Open
#156,532 2 comments 2 reactions 0 assignees View on GitHub
A-cdc C-enhancement T-cdc
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Describe the problem**

Currently, any query parameters required by the user need to be specified when creating the external connection. This makes sense of query parameters related to connecting to the external resource (such as authentication parameters).

However, the `kafka` sink used by `CHANGEFEED` uses query parameters to specify the topic:

- `schema_topic`
- `topic_name`
- `topic_prefix`

These options are interesting, because on the one hand they are indeed about the ultimate destination that the data is sent to. For instance, you can imagine a user who wants to strictly control who can send data to a given topic and does this by controlling permissions on the external connection on that topic.

However, you can also imagine (not so imaginary!) another user who wants 100s of changefeeds to all be able to specify their own explicit topic name but doesn't want to manage 100s of external connections.

One off-the-cuff idea is to allow the sink to override parameters if and only if they weren't already set on the external connection.

But, overall I think we want to consider whether there is anything we can do to improve the UX for the second class of user.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.