microsoft / microsoft/mimalloc

committed > reserved when preallocating huge pages

Open
#1,236 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
13.4k
Forks
1.2k
Avg merge
4d 45m
Merged PRs (30d)
13

Description

When preallocating huge pages, either by `MIMALLOC_RESERVE_HUGE_OS_PAGES` or using `mi_reserve_huge_os_pages*`, I get confusing stats with current version:
```
[2026-02-27T04:50:08.531440Z] subproc 0
[2026-02-27T04:50:08.531529Z] pages peak total current block total#
[2026-02-27T04:50:08.531570Z] touched : 0 0 0 ok
[2026-02-27T04:50:08.531594Z] pages : 61.7 Ki 5.7 Mi 51.4 Ki not all freed
[2026-02-27T04:50:08.531616Z] abandoned : 56.0 Ki 5.5 Mi 45.7 Ki not all freed
[2026-02-27T04:50:08.531638Z] reclaima : 173.3 Ki
[2026-02-27T04:50:08.531660Z] reclaimf : 4.7 Mi
[2026-02-27T04:50:08.531682Z] reabandon : 681.6 Ki
[2026-02-27T04:50:08.531704Z] waits : 1
[2026-02-27T04:50:08.531725Z] extended : 0
[2026-02-27T04:50:08.531747Z] retire : 0
[2026-02-27T04:50:08.531768Z] searches : 2.8 avg
[2026-02-27T04:50:08.531790Z]
[2026-02-27T04:50:08.531812Z] arenas peak total current block total#
[2026-02-27T04:50:08.531833Z] reserved : 21.0 GiB 21.0 GiB 21.0 GiB
[2026-02-27T04:50:08.531857Z] committed : 36.9 GiB 36.9 GiB 36.9 GiB
[2026-02-27T04:50:08.531881Z] reset : 0
[2026-02-27T04:50:08.531903Z] purged : 0
[2026-02-27T04:50:08.531924Z] arenas : 3
[2026-02-27T04:50:08.531946Z] rollback : 0
[2026-02-27T04:50:08.531967Z] mmaps : 45
[2026-02-27T04:50:08.531988Z] commits : 0
[2026-02-27T04:50:08.532010Z] resets : 0
[2026-02-27T04:50:08.532031Z] purges : 0
[2026-02-27T04:50:08.532053Z] guarded : 0
[2026-02-27T04:50:08.532089Z] heaps : 1 1 1
[2026-02-27T04:50:08.532111Z]
[2026-02-27T04:50:08.532133Z] process peak total current block total#
[2026-02-27T04:50:08.532154Z] threads : 180 180 180
[2026-02-27T04:50:08.532176Z] numa nodes: 2
[2026-02-27T04:50:08.532198Z] elapsed : 28488.945 s
[2026-02-27T04:50:08.532219Z] process : user: 1936261.244 s, system: 58167.626 s, faults: 0, peak rss: 3.4 GiB, peak commit: 36.9 GiB
[2026-02-27T04:50:08.532241Z]
```

This shows 39GB committed and 21GB reserved. I definitely not using any additional memory beside reserved 21GB, not `36 - 21 = 15GB` otherwise my RSS would be much larger.

So how am I supposed to get real memory utilization from this? Knowing my program I think it's using 15GB but I can't prove it with code.

Another stat I would like to know is how much of memory is in free lists.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the statistics with MIMALLOC_RESERVE_HUGE_OS_PAGES or mi_reserve_huge_os_pages* and compare the reserved and committed arena values with RSS. Read the statistics output and huge-page allocation behavior first. Done means the reported utilization is explainable and the amount held in free lists is available or clearly accounted for.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.