dotnet / dotnet/runtime

ARM32: Massive /memfd:doublemapper memory leaks causing OOM kills and segfaults during JIT compilation

Open
#121,455 4 comments 0 reactions 1 assignee Claimed by @janvorli View on GitHub
arch-arm32 area-VM-coreclr
Dominant language
C#
Stars
18.3k
Forks
5.6k
PR merge metrics
PR metrics pending

Description

### Description

On ARM32 embedded Linux systems, .NET 8.0.121 exhibits severe memory leaks where `/memfd:doublemapper (deleted)` entries accumulate indefinitely, leading to virtual memory fragmentation, OOM kills, and segmentation faults. This appears to be the same issue described in #89776.

Process memory mappings show thousands of leaked doublemapper entries:
```
0x91287000 0x91297000 0x10000 0x1c31000 /memfd:doublemapper (deleted)
0x91297000 0x912a7000 0x10000 0x1c21000 /memfd:doublemapper (deleted)
0x912a7000 0x912b7000 0x10000 0x1c11000 /memfd:doublemapper (deleted)
... [thousands more entries]
```
Crash Symptoms
1. **Segmentation faults** during JIT compilation in fragmented memory regions
2. **OOM kills** when virtual address space is exhausted
3. **Stack corruption** during exception handling due to memory fragmentation

### Reproduction Steps

https://github.com/ifv-sel/Doublemapper

### Expected behavior

- JIT code pages should be properly unmapped when no longer needed
- `/memfd:doublemapper (deleted)` entries should not accumulate
- System should handle large file operations without memory exhaustion

### Actual behavior

- Doublemapper entries accumulate indefinitely
- Virtual address space becomes heavily fragmented
- Process crashes or gets OOM killed during normal operations

### Regression?

Haven't tested earlier than 8.0.4.07

### Known Workarounds

The following environment variables prevent doublemapper accumulation:

```bash
export DOTNET_TieredCompilation=0 # Primary fix - prevents recompilation
export DOTNET_ReadyToRun=0 # Forces consistent JIT path
export DOTNET_gcServer=0 # Reduces memory overhead
export DOTNET_JitMinOpts=1 # Minimal JIT optimizations
export DOTNET_EnableWriteXorExecute=0 # Bypasses doublemapper entirely
```

### Configuration

- **Platform**: ARM32 (socfpga-cyclone5 - Intel Cyclone V FPGA SoC)
- **OS**: Embedded Linux
- **.NET Version**: 8.0.121
- **Architecture**: ARM32
- **Memory**: Constrained embedded environment

### Other information

Stack Traces

JIT Compilation Crash
```
#0 0xb6859320 in ClassLoader::LoadConstructedTypeThrowing at clsload.cpp:965
#1 0xb685a3d4 in ClassLoader::LoadGenericInstantiationThrowing at clsload.cpp:1724
...
#29 0xb6a14036 in ThePreStub () at arm/asmhelpers.S:199
#30 0x93254004 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
```

Exception Handling Crash
```
#0 IsFunctionFragment (baseAddress=2449276928, pFunctionEntry=0x0) at codeman.cpp:831
#1 FindRootEntry (baseAddress=2449276928, pFunctionEntry=) at codeman.cpp:939
...
#21 0x91fd4034 in ?? () # JIT-compiled code region
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.