cockroachdb / cockroachdb/cockroach
raft: make RawNode aware of log compactions
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
Today, `RawNode` does not proactively discover the fact that the log has been compacted. Instead, it learns this reactively in [maybeSendAppend](https://github.com/cockroachdb/cockroach/blob/0ce38176c1c290f5d3208ad53434d4eece088230/pkg/raft/raft.go#L696-L708) when it tries to fetch entries from log storage.
We should instead notify raft about log compaction explicitly, e.g. via some `Step(MsgLogCompacted)` or `RawNode.LogCompacted(...)` method. Doing this has a few benefits.
**One**. At the point when the `RawNode` is notified about the compaction, it can discover that it can no longer catch up some followers via `MsgApp` (i.e. when `Next` is below the compaction point). So it can enter `StateSnapshot` immediately. Today, this will only happen some time later, when this node tries to send a `MsgApp` again, which in some cases (`StateProbe`, `StateReplicate` with saturated in-flight limits) will happen only in ~heartbeat timeout time.
**Two**. The `raft.Storage` [API](https://github.com/cockroachdb/cockroach/blob/0ce38176c1c290f5d3208ad53434d4eece088230/pkg/raft/storage.go#L28-L40), as well as `raft` code, have to support the `ErrCompacted` returning and handling, because `RawNode` doesn't have an up-to-date knowledge about the log state and can attempt to fetch misaligned log slices. With log compaction notifications, we can get rid of this API entirely and have `RawNode` always request log slices within the correct bounds.
**Three**. Entering the `StateSnapshot` early communicates the intent to send a snapshot to the upper layer. This approach will allow mediating snapshot sending in RACv2 more gracefully than if we implement it today.
Jira issue: CRDB-42838
Contributor guide
Assessment
This issue has not been assessed yet.