cockroachdb / cockroachdb/cockroach
logstore: improve LoadEntries cache usage
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
Current logic in [logstore.LoadEntries](https://github.com/cockroachdb/cockroach/blob/52acfedd49f5c131bc7d493239cc24a38f6e77b9/pkg/kv/kvserver/logstore/logstore.go#L650):
1. Try to load a prefix from the log cache.
2. Load the remainder from storage. While doing so, expand sideloaded entries to account for their size.
This can be improved. Example: the cache has entries `(100, 200]`, and we are trying to load `(50, 150]` or `(50, 250]`. With the current code, the cache will not be used even though half the entries are there.
Improved logic:
1. Load all the cached entries in the requested span.
2. Load the prefix and the suffix from storage.
For (2), we need to use [raftlog.Iterator](https://github.com/cockroachdb/cockroach/blob/52acfedd49f5c131bc7d493239cc24a38f6e77b9/pkg/kv/kvserver/raftlog/iterator.go#L47) directly, and `SeekGE` to the suffix (if any) after having read the prefix (if any).
Incidentally (not really 😀), while doing this optimization, we can remove the one-off [cache accesses](https://github.com/cockroachdb/cockroach/blob/52acfedd49f5c131bc7d493239cc24a38f6e77b9/pkg/kv/kvserver/logstore/sideload.go#L166-L171) while [scanning](https://github.com/cockroachdb/cockroach/blob/52acfedd49f5c131bc7d493239cc24a38f6e77b9/pkg/kv/kvserver/logstore/logstore.go#L704-L712) log storage because all the cached entries are already exhausted. The additional benefit here will be that the log storage scanning code can be decoupled from the cache: we read from the cache before scanning, and maybe update it after. As a result, we will have a "pure storage" version of the `LoadEntries` call.
Part of #136109
Contributor guide
Assessment
This issue has not been assessed yet.