cockroachdb / cockroachdb/cockroach
kvserver: consolidate raft log storage logic
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
The raft log storage logic is scattered across the codebase:
- The `logstore` [package](https://github.com/cockroachdb/cockroach/blob/6a7cdb8d72e78f9afb295c80cea48b6751271501/pkg/kv/kvserver/logstore/logstore.go#L158).
- The `replica_raftlog.go` [file]( https://github.com/cockroachdb/cockroach/blob/6a7cdb8d72e78f9afb295c80cea48b6751271501/pkg/kv/kvserver/replica_raftlog.go#L28) implementing the `raft.LogStorage` interface.
- A few places that synthesize/write the raft state to the storage directly, such as applying a [snapshot](https://github.com/cockroachdb/cockroach/blob/6a7cdb8d72e78f9afb295c80cea48b6751271501/pkg/kv/kvserver/replica_raftstorage.go#L589-L641), replica [destruction](https://github.com/cockroachdb/cockroach/blob/6a7cdb8d72e78f9afb295c80cea48b6751271501/pkg/kv/kvserver/kvstorage/destroy.go#L104), splits/merges.
It would be good to consolidate this code and raft log state management in one package, before doing #8979 and separate raft log.
Unstructured list of `Replica` things that belong to a separate log storage type:
- [x] Log storage [state](https://github.com/cockroachdb/cockroach/blob/ae7600a78eed475957385b634d3ddf3ebc95a94c/pkg/kv/kvserver/replica.go#L592-L626): #145585
- [ ] [localMsgs](https://github.com/cockroachdb/cockroach/blob/ae7600a78eed475957385b634d3ddf3ebc95a94c/pkg/kv/kvserver/replica.go#L480-L493): the write acks from log storage to raft
- [ ] [pendingLogTruncations](https://github.com/cockroachdb/cockroach/blob/ae7600a78eed475957385b634d3ddf3ebc95a94c/pkg/kv/kvserver/replica.go#L1076-L1080) and generally the log truncation machinery (loosely or tightly coupled)
Jira issue: CRDB-44895
Contributor guide
Research direction
Read pkg/kv/kvserver/logstore/logstore.go and replica_raftlog.go first, then trace the direct raft-state writes in replica_raftstorage.go and kvstorage/destroy.go, including snapshot, split, and merge paths. The work is done when raft log storage and state management are consolidated in one package, with localMsgs and pendingLogTruncations addressed as described.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, databases, distributed-systems
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100