cockroachdb / cockroachdb/cockroach
CDC: support resolved messages with db-level feeds for pubsub and kafka
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
*Description*
All db-level changefeeds set split column families which is not compatible with resolved for pubsub and kafka.
See this code in changefeed_stmt:
if opts.IsSet(changefeedbase.OptResolvedTimestamps) &&
opts.IsSet(changefeedbase.OptSplitColumnFamilies) {
return errors.Newf("Resolved timestamps are not currently supported with %s for this sink"+
" as the set of topics to fan them out to may change. Instead, use TABLE tablename FAMILY familyname"+
" to specify individual families to watch.", changefeedbase.OptSplitColumnFamilies)
}
To fix, we'd need to make changes to the topic namer to dynamically generate a list of topics.
Then we'd be able to add coverage more coverage with DB-level feeds metamorphic testing.
We may also be able to add this ability for table level feeds.
Jira issue: CRDB-57276
Epic CRDB-1421
Contributor guide
Assessment
This issue has not been assessed yet.