microsoft / microsoft/snmalloc

RISC-V Linux: flat pagemap reservation is too large on Sv39

Open
#861 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
2k
Forks
138
Avg merge
11h 19m
Merged PRs (30d)
5

Description

snmalloc 0.7.4 assumes a 48-bit address space on Linux/RISC-V. On an Sv39 riscv64 Linux machine, allocator initialisation fails because the flat pagemap reservation is too large:

Failed to initialise snmalloc.
Trace/breakpoint trap (core dumped)

With the default 48-bit setting, snmalloc tries to reserve a 256 GiB pagemap:

mmap(NULL, 274877906944, PROT_NONE,
     MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = -1 ENOMEM

I tried two compile-time Linux/RISC-V address_bits values:

  • address_bits = 38, matching the existing FreeBSD/RISC-V special case. This works on an Sv39 machine because the pagemap reservation becomes much smaller. However, it is too small to cover valid high user-space mappings on systems with a larger virtual address space, such as Sv48. In that case, snmalloc can see addresses above the configured pagemap range and fail when looking them up. #860

  • address_bits = 47. This covers high user-space mappings better and avoids the too-small-pagemap-range problem, but it still fails on the Sv39 machine for SNMALLOC_CHECK_CLIENT builds. Check-mode enables random_pagemap, and FlatPagemap::init<true>() reserves the required pagemap plus an additional 4 * next_pow2(REQUIRED_SIZE) randomisation area. With 47 address bits, the required pagemap is 128 GiB, so the total reservation becomes 640 GiB:

mmap(NULL, 687194767360, PROT_NONE,
     MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = -1 ENOMEM

Some possible directions seem to be:

  • keep a conservative Linux/RISC-V address_bits value and ensure mappings stay within that range;
  • use a larger address_bits value, but make randomized pagemap initialisation fall back to non-randomized placement when the larger reservation fails;
  • select the effective address-space size based on the actual runtime VA layout;
  • replace or relax the flat pagemap assumption for Linux/RISC-V.

I would appreciate guidance on which approach best matches snmalloc's design expectations.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the allocator-initialisation failure on an Sv39 Linux/RISC-V machine, including an SNMALLOC_CHECK_CLIENT build. Start by tracing the Linux/RISC-V address_bits setting and FlatPagemap::init() with random_pagemap enabled, and review #860. Done should avoid an excessive reservation on Sv39 without breaking valid high mappings on larger virtual address spaces.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, linux
Domain
operating-systems, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.