cockroachdb / cockroachdb/cockroach
kvcoord: mux rangefeed metadata can arrive after events from the partial rangefeed it announces
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Describe the problem**
`muxRangeFeed.startSingleRangeFeed` sends the metadata event only after `stream.start()` succeeds ([dist_sender_mux_rangefeed.go#L230-L236](https://github.com/cockroachdb/cockroach/blob/a7e117882b4356096b3a8ea26046baa7e5d53fce/pkg/kv/kvclient/kvcoord/dist_sender_mux_rangefeed.go#L230-L236)). By that point, `receiveEventsFromNode` may already be delivering that stream's data and checkpoint events to the shared event channel from another goroutine, so consumers can observe events for a partial rangefeed before the metadata that announces it.
`OnMetadata`'s documented contract says the callback "occurs when a partial rangefeed begins" ([config.go#L242-L245](https://github.com/cockroachdb/cockroach/blob/a7e117882b4356096b3a8ea26046baa7e5d53fce/pkg/kv/kvclient/rangefeed/config.go#L242-L245)), which implies begins-before-events ordering that the code does not provide. For the logical-data-replication consumer, which uses metadata to install manual split points, a split point can trail data from the child span.
**Suggested fix**
Either hold event delivery for a stream until its metadata has been emitted (e.g. emit metadata as part of the registration step on the receiver path), or explicitly document that metadata ordering relative to the stream's events is unspecified and audit consumers for that assumption.
**Additional context**
Found during an agent-assisted correctness audit of the rangefeed subsystem. Code links are pinned to master @ a7e1178.
Jira issue: CRDB-65659
Contributor guide
Research direction
Start in pkg/kv/kvclient/kvcoord/dist_sender_mux_rangefeed.go, especially muxRangeFeed.startSingleRangeFeed and receiveEventsFromNode, then review the OnMetadata contract in pkg/kv/kvclient/rangefeed/config.go. Trace how metadata and data/checkpoint events reach the shared channel. Done means the ordering contract is made reliable, or the contract is explicitly documented and affected consumers are audited.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100