microsoft / microsoft/mimalloc

mi_reserve_os_memory fails to use 2MB hugepages when reserving >1GB memory

Open
#692 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

stale
Dominant language
C
Stars
13.4k
Forks
1.2k
Avg merge
4d 45m
Merged PRs (30d)
13

Description

Hi,

I am currently using a Linux system where the only hugepage size available is 2MB. With mimalloc 2.0.9, when I tried to use `mi_reserve_os_memory` to reserve 4GB memory, I got the following warning message:
`mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (4294967296 bytes, address: 0x7ffef7200000, alignment: 33554432, commit: 1)`

It appears that `mi_os_get_aligned_hint` always return NULL for `size` > 1GB (I understand this behavior is related to #372). Thus, without an alignment hint, the result of `mi_os_mem_alloc` is unlikely aligned to `MI_SEGMENT_ALIGN` (32MB), which results in the fall back to over-allocation.

Is there a workaround to achieve my goal? Or should I configure the system to enable 1GB hugepages? Thank you!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing mi_reserve_os_memory through mi_os_get_aligned_hint and mi_os_mem_alloc, focusing on the >1GB case and the MI_SEGMENT_ALIGN requirement. Review the related behavior in #372 and reproduce the Linux configuration with only 2MB hugepages; done means determining whether 4GB reservation can avoid over-allocation or documenting the required workaround.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
operating-systems, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.