element-hq / element-hq/synapse

Add metrics for cache "just misses"

Open
#6,400 0 comments 0 reactions 0 assignees View on GitHub
A-Metrics T-Enhancement
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#6400](https://github.com/matrix-org/synapse/issues/6400).

---

Currently its hard to know which caches are too small and would be beneficial to increase. The current metrics for cache hit and miss rates give an indication of how its currently performing, but gives no information on how it would perform if the cache size was increased (e.g. if the cache hit ratio is zero then that could either be because we are always looking up unique items *or* because we're looking up N+1 unique items, in the former case upping the cache size does nothing and in the later upping it would help a lot).

Ideally, we'd have a metric of the cache misses that would have been caught if only the cache was bigger. One way of doing this is caching the last N keys that have been evicted, and during a cache miss also looking up in the evicted key cache.

For large caches this could still be quite a lot of memory, especially if the values are small compared with the key. One potential alternative to the above is to use bloom filters to store the keys that have been recently evicted. These bloom filters can then be rotated (based on age or tracking number of keys added) to give answers like "this cache miss key was *probably* in the cache in the last N entries (or N minutes)".

However, we need to be careful if implementing this that we don't cause performance regressions, as caches are often on hot paths.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.