cockroachdb / cockroachdb/cockroach

util/cache: investigate replacing UnorderedCache with Otter v2

Open
#149,484 0 comments 0 reactions 0 assignees View on GitHub
C-enhancement C-performance
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

[`cache.UnorderedCache`](https://github.com/cockroachdb/cockroach/blob/master/pkg/util/cache/cache.go#L342-L351) is some of the oldest code in CRDB with the base structure of the code dating back to 2014. There has been a lot of [progress on Go caching libraries](https://maypok86.github.io/otter/blog/cache-evolution/) since then. In particular, we should investigate replacing UnorderedCache with [Otter v2](https://maypok86.github.io/otter/blog/cache-evolution/#otter-v2) which benchmarks indicate is significantly faster than other Go caching libraries at read-mostly workloads (which should be true of most/all CRDB uses).

Note that Otter v2 is goroutine safe, while `UnorderedCache` is not and requires the caller to wrap it with a mutex. We'd want to get rid of those mutexes to see the full benefit of Otter v2.

```
$ git grep cache.NewUnorderedCache | grep -v _test.go
ccl/changefeedccl/cdcevent/event.go: eventDescriptorCache := cache.NewUnorderedCache(DefaultCacheConfig)
ccl/changefeedccl/cdcevent/rowfetcher_cache.go: fetchers: cache.NewUnorderedCache(DefaultCacheConfig),
ccl/changefeedccl/cdcevent/rowfetcher_cache.go: fetchers: cache.NewUnorderedCache(DefaultCacheConfig),
ccl/changefeedccl/encoder_avro.go: e.keyCache = cache.NewUnorderedCache(encoderCacheConfig)
ccl/changefeedccl/encoder_avro.go: e.valueCache = cache.NewUnorderedCache(encoderCacheConfig)
ccl/changefeedccl/encoder_json.go: versionCache := cache.NewUnorderedCache(cdcevent.DefaultCacheConfig)
ccl/changefeedccl/encoder_json.go: keySchemaCache: cache.NewUnorderedCache(encoderCacheConfig),
ccl/changefeedccl/encoder_json.go: valueSchemaCache: cache.NewUnorderedCache(encoderCacheConfig),
ccl/changefeedccl/schema_registry.go: src = &schemaRegistryCache{entries: cache.NewUnorderedCache(
ccl/sqlproxyccl/server.go: s.mu.errorLogLimiter = cache.NewUnorderedCache(cacheConfig)
ccl/sqlproxyccl/throttler/local.go: s.mu.throttleCache = cache.NewUnorderedCache(cacheConfig)
sql/closed_session_cache.go: c.mu.data = cache.NewUnorderedCache(cache.Config{
sql/contention/event_store.go: s.mu.store = cache.NewUnorderedCache(cache.Config{
sql/contention/registry.go: txnCache := cache.NewUnorderedCache(txnCacheCfg)
sql/contention/registry.go: txnCache = cache.NewUnorderedCache(txnCacheCfg)
sql/contention/registry.go: internalCache: cache.NewUnorderedCache(cache.Config{
sql/contention/registry.go: txnCache := cache.NewUnorderedCache(txnCacheCfg)
sql/sem/tree/regexp_cache.go: cache: cache.NewUnorderedCache(cache.Config{
sql/sqlliveness/slstorage/slstorage.go: s.mu.liveSessions = cache.NewUnorderedCache(cacheConfig)
sql/sqlliveness/slstorage/slstorage.go: s.mu.deadSessions = cache.NewUnorderedCache(cacheConfig)
sql/sqlstats/insights/store.go: s.mu.insights = cache.NewUnorderedCache(config)
sql/stats/stats_cache.go: tableStatsCache.mu.cache = cache.NewUnorderedCache(cache.Config{
sql/txn_fingerprint_id_cache.go: b.mu.cache = cache.NewUnorderedCache(cache.Config{
util/metric/aggmetric/agg_metric.go: cacheStorage := cache.NewUnorderedCache(cache.Config{
util/tochar/cache.go: ret.mu.cache = cache.NewUnorderedCache(cache.Config{
```

Jira issue: CRDB-52159

Contributor guide

Open the contributing guide

Research direction

Start with pkg/util/cache/cache.go, the linked Otter v2 cache-evolution benchmarks, and the git grep results for cache.NewUnorderedCache. Compare the listed call sites and their mutex usage, then benchmark representative read-mostly workloads. Done means a justified replacement decision, with any required call-site changes and mutex removal identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.