cockroachdb / cockroachdb/cockroach
kvserver,admission: sep-raft-log: revisit store admission control
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
Store admission control (AC) shapes incoming writes/ingests based on various potential bottlenecks (a) flush throughput, (b) L0=>Lbase compaction bandwidth, (c) disk bandwidth (for elastic work). With physical raft log separation https://github.com/cockroachdb/cockroach/issues/16624, the Raft and State machine engines will share the same disk, but be separate wrt flushes and compactions, so the current AC behavior will need to be adjusted.
Since AC modeling and shaping operates at 15s intervals, it currently assumes that the lag from raft log write to state machine application is minimal (compared to the 15s interval). Since that lag is unaffected by raft and state machine separation, we can continue with that assumption. Similarly, we can stay with the current model where the actual shaping is applied when work is being presented for raft log writing. The changes will need to localized to how we do the modeling and how we compute the byte tokens.
Observations:
- Disk bandwidth: AC currently models the write amp for the engine, which will need to be modeled separately for the two engines. Most bandwidth consumption will be from the state machine engine: compactions, flushes. Raft engine will almost never compact (very low write amp) and most flushes will end up dropping most data (due to raft log truncation).
- L0 sub-levels: high sub-level count in the raft log engine is not detrimental to the system, so we could choose to ignore it.
- Flush throughput: Flush throughput will typically be more of a bottleneck for the state machine, since we will not be flushing much from the raft log engine. But it is simple enough to model both engine's peak flush throughput and use the minimum.
Jira issue: CRDB-56191
Contributor guide
Assessment
This issue has not been assessed yet.