FEX-Emu / FEX-Emu/FEX

x64 munmap starting below 4GB returns EOVERFLOW if the range crosses 4GB

Open
#5,847 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
8k
Forks
351
Avg merge
12h 31m
Merged PRs (30d)
102

Description

I think GuestMunmap is wrong for 64-bit guests.

If addr is below 4GB it always goes through MemAllocator32Bit::Munmap, and that function bails with EOVERFLOW whenever addr+length doesn't fit in a uint32. That's not what Linux does. munmap on x86_64 is fine with a range that crosses 4GB; pages that aren't mapped just stay unmapped.

I ran into this with the official amd64 Spotify client (1.2.95.453, CEF). Chromium partition_alloc's FreePagesInternal munmaps a chunk and then immediately int3/ud2 if munmap fails, so you just get a SIGTRAP and no useful errno in the UI. The call that blew up was:

munmap(0x10000, 0x1000ff000) -> -1, errno 75 (EOVERFLOW)

0x10000 + 0x1000ff000 is a little past 4GB, which is exactly the overflow check.

FEX-2608 on Fedora 46 aarch64 (Qualcomm Oryon), Fedora default.erofs rootfs, thunks off.

Small repro, x86_64 binary under FEX:

```c
#define _GNU_SOURCE
#include
#include
#include
#include

int main(void) {
void *addr = (void *)0x10000;
size_t len = 0x1000ff000UL;
int r = munmap(addr, len);
printf("munmap(%p, 0x%zx) = %d errno=%d (%s)\n",
addr, len, r, errno, strerror(errno));
return r == 0 ? 0 : 1;
}
```

Native x86_64 returns 0. Under FEX I get -1 / EOVERFLOW.

The check lives in MemAllocator32Bit::Munmap (LinuxAllocator.cpp). GuestMunmap is what routes low addresses there unconditionally (SyscallsSMCTracking.cpp). Weirdly, 64-bit mmap only uses that allocator if MAP_32BIT is set, but munmap uses it for any addr < 4GB even when the length clearly isn't a 32-bit mapping.

This isn't #5453. Spotify still needs --no-zygote because of the vfork/posix_spawn thing. Once that's out of the way it dies here instead.

Contributor guide

Open the contributing guide

Research direction

Start with GuestMunmap in SyscallsSMCTracking.cpp and MemAllocator32Bit::Munmap in LinuxAllocator.cpp. Run the provided x86_64 repro under FEX and compare it with native x86_64 behavior. Done means a low-address munmap range crossing 4GB succeeds like Linux instead of returning EOVERFLOW.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, linux
Domain
operating-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.