cockroachdb / cockroachdb/cockroach

storage,kv: consider reorganizing local keyspace keys to reduce w-amp

Open
#118,762 0 comments 0 reactions 0 assignees View on GitHub
A-kv A-storage C-enhancement T-storage
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Within the range local keyspace, we have some keys that are unversioned (eg, RaftHardState, RaftAppliedState, RaftTruncatedState) but frequently written. With expiration-based leases, the lease key is also very frequently written. Today these keys are not stored continuously; replica raft logs, replicated shared locks, etc are interspersed. This interspersing increases write amplification.

Consider if we stored all the lease keys together in a continuous lease key prefix and imposed sstable boundary guards (cockroachdb/pebble#517) on either side. With expiration-based leases enabled, every memtable flush would typically write 1 lease key per replica to L0. Due to sstable boundary guards, this L0 file should be able to move-compacted to L5 and then compacted to L6 completely replacing the contents of the L6 sstable. This would reduce the write amp for the lease keys themselves to 3 (to WAL, to L0, to L6).

Other high-write, unversioned local keys might also benefit from a similar approach.

I'm uncertain the w-amp savings are enough to warrant the complexity of a migration.

Jira issue: CRDB-35891

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.