microsoft / microsoft/snmalloc
Large OS pages support
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 2k
- Forks
- 138
- Avg merge
- 11h 19m
- Merged PRs (30d)
- 5
Description
Hi,
I'm trying to make snmalloc use large OS pages in my application. My use case has been mentioned in https://github.com/microsoft/snmalloc/issues/484#issuecomment-1073236187. My application is targeting Windows. It is memory bottlenecked and I want to reduce page faults. On Linux, 4KB pages can be merged into 2MB large pages implicitly and automatically, but on Windows large page allocation is explicit.
I have tried to modify the PAL to use large pages. The key modifications are:
static SNMALLOC_CONSTINIT_STATIC size_t minimum_alloc_size = 0x200000; // 2MB
static constexpr size_t page_size = 0x200000; // 2MB
// in `notify_using`
void* r = VirtualAlloc(p, size, MEM_COMMIT | MEM_LARGE_PAGES, PAGE_READWRITE);
This PAL works well with snmalloc1, except the need of some small patches (alignas(OS_PAGE_SIZE) not possible, and need to check rsize > OS_PAGE_SIZE before rsize - OS_PAGE_SIZE). I don't know there are flaws or not but it can just run.
snmalloc2 cannot work with this PAL. It crashes at:
https://github.com/microsoft/snmalloc/blob/26324e8bfc28eda735a3335f83f3f1d8f21cc4a0/src/mem/corealloc.h#L690
I'm wondering whether I can use PAL to enable large os page support and expect snmalloc1 to work perfectly, and why this don't work with snmalloc2.
Sorry, may be duplicate with #222
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the PAL changes in commit 662e209a9528250e19f02a71e3b9cb5b3c7a5172 and the crash at src/mem/corealloc.h line 690. Compare the large-page behavior between snmalloc1 and snmalloc2, including the small patches noted in the issue. Done means documenting whether the PAL can support large pages reliably and why snmalloc2 fails.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- operating-systems, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100