microsoft / microsoft/snmalloc

Large OS pages support

Open
#485 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement windows
Dominant language
C++
Stars
2k
Forks
138
Avg merge
11h 19m
Merged PRs (30d)
5

Description

Hi,
I'm trying to make snmalloc use large OS pages in my application. My use case has been mentioned in https://github.com/microsoft/snmalloc/issues/484#issuecomment-1073236187. My application is targeting Windows. It is memory bottlenecked and I want to reduce page faults. On Linux, 4KB pages can be merged into 2MB large pages implicitly and automatically, but on Windows large page allocation is explicit.

I have tried to modify the PAL to use large pages. The key modifications are:

static SNMALLOC_CONSTINIT_STATIC size_t minimum_alloc_size = 0x200000;    // 2MB
static constexpr size_t page_size = 0x200000;                             // 2MB

// in `notify_using`
void* r = VirtualAlloc(p, size, MEM_COMMIT | MEM_LARGE_PAGES, PAGE_READWRITE);

This PAL works well with snmalloc1, except the need of some small patches (alignas(OS_PAGE_SIZE) not possible, and need to check rsize > OS_PAGE_SIZE before rsize - OS_PAGE_SIZE). I don't know there are flaws or not but it can just run.

snmalloc2 cannot work with this PAL. It crashes at:
https://github.com/microsoft/snmalloc/blob/26324e8bfc28eda735a3335f83f3f1d8f21cc4a0/src/mem/corealloc.h#L690

I'm wondering whether I can use PAL to enable large os page support and expect snmalloc1 to work perfectly, and why this don't work with snmalloc2.


Sorry, may be duplicate with #222

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the PAL changes in commit 662e209a9528250e19f02a71e3b9cb5b3c7a5172 and the crash at src/mem/corealloc.h line 690. Compare the large-page behavior between snmalloc1 and snmalloc2, including the small patches noted in the issue. Done means documenting whether the PAL can support large pages reliably and why snmalloc2 fails.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
operating-systems, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.