cockroachdb / cockroachdb/cockroach
kvserver: reduce usage of indexed batches
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
The state machine apply stack currently [uses](https://github.com/cockroachdb/cockroach/blob/c157bb60ff1639e0cf2105aa13ee5f688c527680/pkg/kv/kvserver/replica_application_state_machine.go#L146) `Engine.NewBatch()` (indexed batch, which can read its own writes), to accommodate a wide spectrum of commands which may or may not need to read. However, indexed batches are more expensive than unindexed or write-only batches.
For the most part, the leaseholder has done all the preparation at evaluation time, and the replica just needs to [blind-write](https://github.com/cockroachdb/cockroach/blob/3b44c5221318df22c1b568fe50869deb0bf64067/pkg/kv/kvserver/app_batch.go#L141) the batch. Even if a command application needs to read, it can do so from a separate `Reader` since at apply time we are holding `raftMu`. There are probably close to / zero cases when we actually need to read own writes.
One notable reason why we need a readable batch is [this](https://github.com/cockroachdb/cockroach/blob/47a42278ac490ed03eb1d4f8a6dd70f854881a1f/pkg/kv/kvserver/replica_app_batch.go#L714-L717) `SetRangeAppliedState` which uses `MVCCPutProto`, which needs to read in order to maintain MVCC stats. There are multiple similar read-then-write cases.
As an example, the raft log truncations (after #144615) can be carried out with only the `Writer` interface.
Jira issue: CRDB-49463
Contributor guide
Research direction
Start with the apply stack in pkg/kv/kvserver/replica_application_state_machine.go and the blind-write path in pkg/kv/kvserver/app_batch.go. Then inspect SetRangeAppliedState and related read-then-write cases in pkg/kv/kvserver/replica_app_batch.go, along with the raft log truncation changes referenced by the issue. Done means indexed batches are avoided where separate reads or write-only operations suffice, while required read-then-write behavior remains correct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, distributed-systems, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100