cockroachdb / cockroachdb/cockroach
storage,kv: consider reorganizing local keyspace keys to reduce w-amp
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
Within the range local keyspace, we have some keys that are unversioned (eg, RaftHardState, RaftAppliedState, RaftTruncatedState) but frequently written. With expiration-based leases, the lease key is also very frequently written. Today these keys are not stored continuously; replica raft logs, replicated shared locks, etc are interspersed. This interspersing increases write amplification.
Consider if we stored all the lease keys together in a continuous lease key prefix and imposed sstable boundary guards (cockroachdb/pebble#517) on either side. With expiration-based leases enabled, every memtable flush would typically write 1 lease key per replica to L0. Due to sstable boundary guards, this L0 file should be able to move-compacted to L5 and then compacted to L6 completely replacing the contents of the L6 sstable. This would reduce the write amp for the lease keys themselves to 3 (to WAL, to L0, to L6).
Other high-write, unversioned local keys might also benefit from a similar approach.
I'm uncertain the w-amp savings are enough to warrant the complexity of a migration.
Jira issue: CRDB-35891
Contributor guide
Assessment
This issue has not been assessed yet.