cockroachdb / cockroachdb/cockroach
raft: make MsgApp.Entries in StateProbe empty
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
When the follower replication flow is in `StateProbe`, the leader will likely send `MsgApp`s [with log entries](https://github.com/cockroachdb/cockroach/blob/d0dbf6a258c0bdc66128e412f32ff6e490f33b7b/pkg/raft/tracker/progress.go#L195).
The downsides of this:
- the probes aren't subject to replication admission control, and can lead to overload
- the probes are constructed inside `RawNode` under `Replica.mu`, so `MsgApp` construction with entries may lead to IO under this mutex. We would like to minimize it: #130955.
A simple fix is to make `StateProbe` messages outright empty. However, this would have a performance impact because it's optimized for the happy case when the first probe is successful (and so some entries are replicated immediately). In a general case, this reduces post-`StateProbe` replication latency by one roundtrip to the follower.
There are some ideas in https://github.com/etcd-io/raft/issues/150 and #136296 on how to optimize these roundtrips differently. The "hints" in `MsgAppResp` can be extended to carry more information, as well as embedded in the `MsgVoteResp` to eliminate the first post-election probing roundrip.
Jira issue: CRDB-42653
Contributor guide
Assessment
This issue has not been assessed yet.