cockroachdb / cockroachdb/cockroach

streamingccl: zone config streaming follow up tasks

Open
#109,059 1 comment 0 reactions 1 assignee Claimed by @msbutler View on GitHub
A-disaster-recovery C-wishlist T-disaster-recovery
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Once the span config replication stream is all hooked up in #106823, a few follow up tasks should be considered (in order of priority):

- [x] Consider adding some internal retry behavior in the span config event stream
- [x] Remove the `SpanConfigStreamClient` from the `StreamClient` interface. The `SpanConfigStreamClient` only requires the `SetupSpanConfigStream()` and the `Ping()` call. Nothing else.
- [ ] Deduplicate code in `span_config_event_stream.go` and `event_stream.go`
- [x] Consider creating a new [StreamPartitionSpec](https://github.com/cockroachdb/cockroach/pull/108356/files#r1297801266) for the Span Config stream in `stream.proto`
- [ ] Pull the span config deduplication logic out of the `streamEventBatcher` to avoid to 2 loops over the fresh rangefeed flush. See [here](https://github.com/cockroachdb/cockroach/pull/108356/files#r1297812992).
- [ ] Consider removing the checkpoint pacer on span config event flushing. More [here](https://github.com/cockroachdb/cockroach/blob/master/pkg/ccl/streamingccl/streamproducer/span_config_event_stream.go#L204).
- [ ] Clean up logic around replicating ephemeral system tables. they currently get skipped due to the rekeyer.
- [ ] Crop the span the rangefeed watches to the replicating tenant's span. It currently watches the whole system tenant's span config table, and the span config event stream filters out irrelevant updates.
- deprioritized for now, as the api for finding the exact span is clunky and schema dependent, and because the rangefeed will have to filter out keys anyway, as it surfaces all updates in a range.
- [ ] Flush to the destination side without holding up the the rangfeed.
- deprioritized, until the event_stream conducts async flushing.

Jira issue: CRDB-30756

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.