microsoft / microsoft/mimalloc
Faithful, performant tracking of committed memory and limit enforcement
- Dominant language
- C
- Stars
- 13.4k
- Forks
- 1.2k
- Avg merge
- 4d 45m
- Merged PRs (30d)
- 13
Description
**Problem statement:** We want to use mimalloc for an analytical database on Linux. We deploy the database in Kubernetes pods with a memory limit. If the process requests more memory than this limit, Kubernetes kills it. We want to avoid those out-of-memory kills and instead gracefully recover inside our database server.
**Potential solution:** To avoid our process getting killed, we want to track the amount of memory held by the process and fail allocations before requesting more memory from the operating system when the process is close to its limit.
At a high level, we need:
1. mimalloc to provide an accurate measure of its OS-backed memory and
2. a way for the embedding application to keep that measure below a limit.
Towards those goals, we propose two independent changes:
1. a mode in which committed follows physical memory with performance being on par with stock mimalloc
2. and paired callbacks through which the application can reject increases and observe decreases.
We have an in-house prototype of those changes, described in more detail below. We are happy to contribute those changes upstream, but before opening PRs would appreciate your feedback on our design thoughts. We would also be interested in any existing alternatives, prior discussions, configuration knobs, etc. which we may have missed.
**Keeping committed aligned with RSS**
We would like to use mimalloc's committed-memory statistic to track our RSS.
We measured our database on TPC-DS SF1000 using mimalloc 3.4.1 with `MIMALLOC_ARENA_EAGER_COMMIT=0.`
In a normal release build, the mimalloc committed counter (green) behaves like a high-water mark instead of following the process RSS (black).
This happens because `mi_os_decommit_ex` subtracts from committed only when `_mi_prim_decommit` returns `needs_recommit == true`. In a normal Linux release build, `_mi_prim_decommit` releases the physical pages with `madvise(MADV_DONTNEED)` but returns `needs_recommit == false`. `mi_arena_purge` therefore retains the slices_committed bits, and a later allocation in `mi_arena_try_alloc_at` call `_mi_os_reuse` instead of `_mi_os_commit_ex`.
The existing Linux implementation sets needs_recommit to true when mimalloc is built with `MI_SECURE > 2`. We repeated the same benchmark and runtime configuration with mimalloc built using `MI_SECURE=4` and `MI_FREE_IS_CHECKED=1`. In this build, committed closely tracks RSS.
In those builds, mimalloc's committed-memory statistic closely tracks RSS.
However, it comes at a high performance cost. `_mi_prim_decommit` calls `mprotect(PROT_NONE)`, and `_mi_prim_commit` later calls `mprotect(PROT_READ|PROT_WRITE)`. These calls update the protection attributes of the affected Linux virtual memory areas (VMAs) and the corresponding page-table entries on every purge and recommit cycle. The full sequential benchmark recorded about 2.3 million additional `mprotect` calls, and the peak VMA count increased from 16,472 to 49,781. In a separate targeted benchmark, this led to a median runtime increase from 284s to 292 seconds, by 8s, i.e. 2.7%.
We want mimalloc's committed-memory statistic to closely track RSS, but don't want to pay this performance penalty.
We therefore propose a configuration that provides faithful committed-memory accounting without the overhead of calling mprotect. Our prototype always sets needs_recommit to true but omits the mprotect calls.
**Enforcing a limit on committed memory**
The embedding application needs to enforce a limit against the same accounting. We propose paired callbacks around changes to committed. Our prototype adds `mi_commit_callback_fun(size_t bytes, void* arg)` and `mi_decommit_callback_fun(size_t bytes, void* arg)`, registered together through mi_register_rss_callbacks.
Before every increase to committed, the commit callback atomically reserves the corresponding number of bytes. If it returns false, mimalloc abandons the attempted increase and reports the allocation as failed. After every decrease to committed, the decommit callback releases those bytes. Bracketing every change to committed this way allows the application to maintain a race-free atomic counter across allocator threads.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the Linux paths around mi_os_decommit_ex, _mi_prim_decommit, _mi_prim_commit, mi_arena_purge, and mi_arena_try_alloc_at. Review the proposed needs_recommit behavior and mi_register_rss_callbacks API, along with the reported benchmark results. Done means reaching maintainer agreement on faithful committed-memory accounting and paired limit-enforcement callbacks before implementation begins.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, linux
- Domain
- operating-systems, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100