microsoft / microsoft/mimalloc
mi_reserve_os_memory fails to use 2MB hugepages when reserving >1GB memory
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 13.4k
- Forks
- 1.2k
- Avg merge
- 4d 45m
- Merged PRs (30d)
- 13
Description
Hi,
I am currently using a Linux system where the only hugepage size available is 2MB. With mimalloc 2.0.9, when I tried to use `mi_reserve_os_memory` to reserve 4GB memory, I got the following warning message:
`mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (4294967296 bytes, address: 0x7ffef7200000, alignment: 33554432, commit: 1)`
It appears that `mi_os_get_aligned_hint` always return NULL for `size` > 1GB (I understand this behavior is related to #372). Thus, without an alignment hint, the result of `mi_os_mem_alloc` is unlikely aligned to `MI_SEGMENT_ALIGN` (32MB), which results in the fall back to over-allocation.
Is there a workaround to achieve my goal? Or should I configure the system to enable 1GB hugepages? Thank you!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing mi_reserve_os_memory through mi_os_get_aligned_hint and mi_os_mem_alloc, focusing on the >1GB case and the MI_SEGMENT_ALIGN requirement. Review the related behavior in #372 and reproduce the Linux configuration with only 2MB hugepages; done means determining whether 4GB reservation can avoid over-allocation or documenting the required workaround.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- operating-systems, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100