dragonflydb / dragonflydb/dragonfly
Memory accounting code: review
- Dominant language
- C++
- Stars
- 31.5k
- Forks
- 1.3k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 121
Description
I've been re-reading our memory accounting code to add better heuristics:
- [x] `max_memory_limit` is just a global `size_t` that can be mutably changed by the config registry 🎖
- [x] `used_mem_peak` and `rss_mem_peak` are unnecessarily global
- [x] Deny OOM:
* `GetMemoryUsage` updates every 1us, whereas global atomics are updated every ~`1/shard_count`**ms** (rss even less frequently), i.e. 99% of the calls are ineffective
* Not sure why we use command timestamps?
* No uniformity in flag access. Some code calls `GetFlag` on hot paths, server state caches at least a handful of other flags. Once calculated parameters were left forgotten with flag updates (like `soft_budget_limit_`)
* "Modules" often don't care about centralizing memory calculations. Deny oom has 3 places where it repeats its calculation (using different sources at the same time), tiering calculates it's `offload threshold` two times - same formula - differently looking code
* No real comments to clarify the meaning or usage of important variables or calculations
Contributor guide
Assessment
This issue has not been assessed yet.