cockroachdb / cockroachdb/cockroach
changefeedccl: potential duplicate resolved messages when resolved=0s
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
When the resolved option is set to 0 (which is the default value), duplicate resolved messages can be sent if the highwater is not advancing.
This is because the logic for checking if we can emit another resolved timestamp will check 1) that we've checkpointed progress and 2) that the resolved timestamp has advanced by at least the option value (0 in this case). The first case will true if we've created a span-level checkpoint even if the highwater has not advanced.
Code pointers:
* Checking if progress has been checkpointed
https://github.com/cockroachdb/cockroach/blob/a64757f26ef6c8dc9bb84f1c9cb2f9da49ce962f/pkg/ccl/changefeedccl/changefeed_processors.go#L1737
* Checking if resolved timestamp has advanced by option value
https://github.com/cockroachdb/cockroach/blob/a64757f26ef6c8dc9bb84f1c9cb2f9da49ce962f/pkg/ccl/changefeedccl/changefeed_processors.go#L2224
Repro steps:
```
SET CLUSTER SETTING changefeed.span_checkpoint.interval = '1ms';
SET CLUSTER SETTING changefeed.span_checkpoint.lag_threshold = '1ms';
CREATE TABLE large (id PRIMARY KEY) AS SELECT generate_series(1, 100000);
ALTER TABLE large SPLIT AT SELECT id FROM large ORDER BY random() LIMIT 5;
CREATE CHANGEFEED FOR large WITH initial_scan ='no', resolved;
{"key":"NULL","table":"NULL","value":"{\"resolved\":\"1757692330279202000.0000000000\"}"}
{"key":"NULL","table":"NULL","value":"{\"resolved\":\"1757692330279202000.0000000000\"}"}
{"key":"NULL","table":"NULL","value":"{\"resolved\":\"1757692330279202000.0000000000\"}"}
{"key":"NULL","table":"NULL","value":"{\"resolved\":\"1757692330279202000.0000000000\"}"}
{"key":"NULL","table":"NULL","value":"{\"resolved\":\"1757692332000000000.0000000000\"}"}
{"key":"NULL","table":"NULL","value":"{\"resolved\":\"1757692332000000000.0000000000\"}"}
{"key":"NULL","table":"NULL","value":"{\"resolved\":\"1757692332000000000.0000000000\"}"}
{"key":"NULL","table":"NULL","value":"{\"resolved\":\"1757692332000000000.0000000000\"}"}
```
Jira issue: CRDB-54434
Contributor guide
Assessment
This issue has not been assessed yet.