cockroachdb / cockroachdb/cockroach

crosscluster/logical: transactional LDR applier races input.Next() between reader goroutine and DrainHelper

Open
#172,370 3 comments 0 reactions 1 assignee Claimed by @DarrylWong View on GitHub
A-cdc A-logical-data-replication C-bug O-agent T-cdc
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Describe the problem**

The applier consumes its input in a background goroutine (`runInputReader`) but
also registers the same input in `InputsToDrain`. On an internal error (e.g.
writer `ReplicationError` via `errCh`), `Next()` moves to draining and
`DrainHelper` starts calling `input.Next()` while `runInputReader` is still
looping on it — the reader's context is only cancelled later in
`TrailingMetaCallback`. `RowSource.Next` is not safe for concurrent use.

**Impact**

Data race under stats collectors, and row-stealing gives the still-running
Applier a gapped sequence → spurious `missing dependency` assertions during
teardown. (The poisoned frontier cannot be persisted — draining stops frontier
metadata — so this is a race/teardown bug, not checkpoint corruption.)

**Code references**
- [txnmode/ldr_applier_processor.go:97-101](https://github.com/cockroachdb/cockroach/blob/master/pkg/crosscluster/logical/txnmode/ldr_applier_processor.go#L97-L101), [ldr_applier_processor.go:416](https://github.com/cockroachdb/cockroach/blob/master/pkg/crosscluster/logical/txnmode/ldr_applier_processor.go#L416)

**Suggested fix**

`InputsToDrain: nil` with the reader goroutine responsible for draining, or
stop the reader before draining begins.

Epic CRDB-65552

Jira issue: CRDB-65605

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.