cockroachdb / cockroachdb/cockroach
util/cache: investigate replacing UnorderedCache with Otter v2
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
[`cache.UnorderedCache`](https://github.com/cockroachdb/cockroach/blob/master/pkg/util/cache/cache.go#L342-L351) is some of the oldest code in CRDB with the base structure of the code dating back to 2014. There has been a lot of [progress on Go caching libraries](https://maypok86.github.io/otter/blog/cache-evolution/) since then. In particular, we should investigate replacing UnorderedCache with [Otter v2](https://maypok86.github.io/otter/blog/cache-evolution/#otter-v2) which benchmarks indicate is significantly faster than other Go caching libraries at read-mostly workloads (which should be true of most/all CRDB uses).
Note that Otter v2 is goroutine safe, while `UnorderedCache` is not and requires the caller to wrap it with a mutex. We'd want to get rid of those mutexes to see the full benefit of Otter v2.
```
$ git grep cache.NewUnorderedCache | grep -v _test.go
ccl/changefeedccl/cdcevent/event.go: eventDescriptorCache := cache.NewUnorderedCache(DefaultCacheConfig)
ccl/changefeedccl/cdcevent/rowfetcher_cache.go: fetchers: cache.NewUnorderedCache(DefaultCacheConfig),
ccl/changefeedccl/cdcevent/rowfetcher_cache.go: fetchers: cache.NewUnorderedCache(DefaultCacheConfig),
ccl/changefeedccl/encoder_avro.go: e.keyCache = cache.NewUnorderedCache(encoderCacheConfig)
ccl/changefeedccl/encoder_avro.go: e.valueCache = cache.NewUnorderedCache(encoderCacheConfig)
ccl/changefeedccl/encoder_json.go: versionCache := cache.NewUnorderedCache(cdcevent.DefaultCacheConfig)
ccl/changefeedccl/encoder_json.go: keySchemaCache: cache.NewUnorderedCache(encoderCacheConfig),
ccl/changefeedccl/encoder_json.go: valueSchemaCache: cache.NewUnorderedCache(encoderCacheConfig),
ccl/changefeedccl/schema_registry.go: src = &schemaRegistryCache{entries: cache.NewUnorderedCache(
ccl/sqlproxyccl/server.go: s.mu.errorLogLimiter = cache.NewUnorderedCache(cacheConfig)
ccl/sqlproxyccl/throttler/local.go: s.mu.throttleCache = cache.NewUnorderedCache(cacheConfig)
sql/closed_session_cache.go: c.mu.data = cache.NewUnorderedCache(cache.Config{
sql/contention/event_store.go: s.mu.store = cache.NewUnorderedCache(cache.Config{
sql/contention/registry.go: txnCache := cache.NewUnorderedCache(txnCacheCfg)
sql/contention/registry.go: txnCache = cache.NewUnorderedCache(txnCacheCfg)
sql/contention/registry.go: internalCache: cache.NewUnorderedCache(cache.Config{
sql/contention/registry.go: txnCache := cache.NewUnorderedCache(txnCacheCfg)
sql/sem/tree/regexp_cache.go: cache: cache.NewUnorderedCache(cache.Config{
sql/sqlliveness/slstorage/slstorage.go: s.mu.liveSessions = cache.NewUnorderedCache(cacheConfig)
sql/sqlliveness/slstorage/slstorage.go: s.mu.deadSessions = cache.NewUnorderedCache(cacheConfig)
sql/sqlstats/insights/store.go: s.mu.insights = cache.NewUnorderedCache(config)
sql/stats/stats_cache.go: tableStatsCache.mu.cache = cache.NewUnorderedCache(cache.Config{
sql/txn_fingerprint_id_cache.go: b.mu.cache = cache.NewUnorderedCache(cache.Config{
util/metric/aggmetric/agg_metric.go: cacheStorage := cache.NewUnorderedCache(cache.Config{
util/tochar/cache.go: ret.mu.cache = cache.NewUnorderedCache(cache.Config{
```
Jira issue: CRDB-52159
Contributor guide
Research direction
Start with pkg/util/cache/cache.go, the linked Otter v2 cache-evolution benchmarks, and the git grep results for cache.NewUnorderedCache. Compare the listed call sites and their mutex usage, then benchmark representative read-mostly workloads. Done means a justified replacement decision, with any required call-site changes and mutex removal identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100