microsoft / microsoft/mimalloc

Not able to prevent additional mmap-s beside reserved memory

Open
#1,228 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
13.4k
Forks
1.2k
Avg merge
4d 45m
Merged PRs (30d)
13

Description

I'm using `mimalloc` as drop-in *malloc* replacement. For my usecase I want to preallocate big area of 2M huge-pages and make `mimalloc` use that area for all allocations unless it's fully used in which case I want to fallback to additional memory (regular `mmap`).

The preallocated area should be enough for 99.9% of runtime and only in extreme cases (e.g. bugs and leaks) I expect additional `mmap`s.

I'm setting these environment options:
```
MIMALLOC_RESERVE_HUGE_OS_PAGES=14
MIMALLOC_PURGE_DELAY=-1
MIMALLOC_VERBOSE=1
MIMALLOC_SHOW_STATS=1
MIMALLOC_SHOW_ERRORS=1
```

I've also added logging around `mmap/munmap` calls. I expected not to see any additional calls besides initial 14 calls to `mmap` with `1 GiB` size. However I still see numerous `mmap` calls with size 64kb at init time.

When I tried to setting `disallow_os_alloc=1`, I've got ENOMEM early in application init, definitely before 14GB were exhausted. I looks like *mimalloc* tries to allocate from OS even if there is plenty of free memory in preallocated area.

Am I misunderstanding something?

There's similar recent question https://github.com/microsoft/mimalloc/issues/1218, but AFAIU it only applies to arenas larger than 16GB while in my case I have 14GB.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the issue with the listed MIMALLOC_* settings, `disallow_os_alloc=1`, and the mmap/munmap logging described in the report. Compare the 64 KiB initialization mappings with the reserved huge-page mappings and determine whether the observed behavior is expected or indicates a bug. Done means the mmap behavior and early ENOMEM result are explained, with a focused regression test or a clearly documented limitation if applicable.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
operating-systems, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.