cockroachdb / cockroachdb/cockroach
storage: allow configuration of memtable size
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
Today a Pebble memtable is always 64MB. The size of the memtable impacts write bandwidth, because the longer a flush is delayed the more likely raft log truncation is able to elide keys before they're written to disk. A similar dynamic applies to expiration-based leases which repeatedly overwrite the same set of keys.
Some customers' environments could easily tolerate significantly larger memtables. Allocation of a new memtable reserves space from the block cache, so as long as the block cache is sufficiently large there is not significant risk exhausting memory.
I'm not sure how we would want to expose this. Some clusters run on heterogeneous hardware, and a particular memtable size might not be appropriate for all stores. Additionally some clusters run many stores per node, so an increase in memtable size is multiplied across the configured stores. Some ability to configure memtable size may make it more feasible for large customers to make use of expiration-based leases.
Jira issue: CRDB-35825
Contributor guide
Assessment
This issue has not been assessed yet.