cockroachdb / cockroachdb/cockroach
Stats cache improvements to support 1M tables
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
In [#171215](https://github.com/cockroachdb/cockroach/issues/171215), we decided to increase the default cache size, but did not otherwise change the cache structure. To support 1M tables, we may have to do something more radical. It's unlikely that 1M tables will result in a working set that fits in memory on a single node, meaning we have to figure out a way to make the entries more compact, reduce the effective working set size or reduce the number of entries a single node needs to cache.
**Reducing Entry Size**
The big driver of entry size is the size of histograms. We've had a number of ideas that attack the size of histograms, including:
1. Caching histograms separately so that we can keep a smaller number of histograms in memory than other table stats. The question here is what criteria to use to bring histograms into memory since it wouldn't help us much to not cache histograms if we're just going to pull them in whenever the planner asks for them.
2. Reducing the resolution of histograms in memory by combining buckets.
3. Disabling histograms when the cache is under memory pressure.
**Reducing Effective Working Set Size**
The required size of the stats cache is driven by the number of tables we need to keep in the cache. This is largely workload dependent, but there are some things we can do:
1. Testing shows that we need approximately 2x the actual working set size to keep the working set in memory when there are periodic accesses to non-working set tables. A scan resistant LRU would reduce this to approximately 1x the working set size.
2. Dynamically sizing the LRU based on the cached lifetime of cached objects. In this case, we set a target time in cache and time since last access for cached objects, which lets the cache discover its own optimal size.
3. Make the planner a bit more selective about when reaches for histograms?
**Reduce Node Working Set Size**
1. Partition the workload so that certain databases and schemas are largely served by a particular set of gateway nodes. This isn't really under our control, but it is something a savvy operator could do to reduce memory pressure.
Jira issue: CRDB-68111
Contributor guide
Assessment
This issue has not been assessed yet.