[ASan][x86-64] Non-PIE binaries larger than ~2 GiB collide with the fixed shadow offset and abort at init
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
At Meta we build much of our C++ as **non-PIC** (and therefore link executables **non-PIE**) to squeeze out some last remaining performance. Our binaries are getting large, and as they approach 2 GiB we're hitting an ASan startup failure that I think can be avoided, though not indefinitely but we can avoid the problem a bit more with 2 solutions I propose at the end.
On Linux/x86-64, AddressSanitizer uses a fixed, compile-time shadow offset of `0x7fff8000` for the default (non-PIE) mapping. This places the application's `LowMem` region at `[0x000000000000, 0x00007fff7fff]` (roughly 2 GiB) with `LowShadow` immediately above it at `0x00007fff8000`.
A **non-PIE** executable is loaded at a low fixed address and grows upward (`.text` / `.rodata` / `.data` / `.bss`). Once its loaded image approaches ~2 GiB, the top of the image runs into `LowShadow` and ASan aborts during initialization:
```
==NNN==Shadow memory range interleaves with an existing memory mapping. ASan cannot proceed correctly. ABORTING.
==NNN==ASan shadow was supposed to be located in the [0x00007fff7000-0x10007fff7fff] range.
```
Building the executable as PIE avoids this (the image loads high, into `HighMem`, at an ASLR base rather than in the low 2 GiB window), but we intentionally avoid PIC/PIE for the performance reasons mentioned at the start.
**Key lines**:
https://github.com/llvm/llvm-project/blob/c4026e4350fd4d70692f205aff8764055a8b4ce1/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp#L105
https://github.com/llvm/llvm-project/blob/c4026e4350fd4d70692f205aff8764055a8b4ce1/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp#L578
Here is a simpler diagram:
```
Low addresses
0x000000000000
|
| Application executable
| fixed address: 0x00200000
|
| .text, .rodata, .data...
|
0x00007fd42000
| .bss
| Application continues growing
|
0x00007fff8000 <----- ASan LowShadow begins
|XXXXXXXXXXXX
|XX COLLISION Application .bss overlaps shadow
|XXXXXXXXXXXX
0x0000813b7000 <----- Application ends
|
| ASan shadow/gap
|
0x10007fff8000 <----- ASan HighMem begins
|
High addresses
```
The instrumentation emits the shadow address as `(mem >> scale) + offset` with `offset` compiled in as an immediate.
Today `0x7fff8000 < 0x80000000`, so it folds into the shadow load as a signed `disp32` (i.e. any offset larger than 2 GiB no longer fits and must be materialized (`movabs` into a register) per access). Notably the runtime's outlined check already has exactly this fast/slow split, so a larger offset is partly anticipated in-tree.
I do see `-mllvm -asan-force-dynamic-shadow` as a potential solution but this is not setup for x86-64 linux.
I see two possible solutions:
1. `-mllvm -asan-mapping-offset`. This also needs the header file to be updated and maybe add missing documentation on this as there is no supported, documented "large offset" configuration. This will also cause an extra register.
* Would be great if this can be configured via `-D` but looks like the code needs to be augmented to support it
2. opt-in dynamic-shadow support for Linux/x86-64. Wire up `ASAN_SHADOW_OFFSET_DYNAMIC` / `__asan_shadow_memory_dynamic_address` for so the shadow can be relocated at runtime to avoid collisions, at the cost of one extrsa load per access. * This could ship as a second runtime variant that users opt into, keeping the fast constant-offset path as the default for small binaries.
My two questions:
* Is there appetite for either approach upstream? Is there prior art, or a specific reason dynamic shadow was never enabled for Linux/x86-64?
* Are there known correctness concerns with a larger fixed offset on Linux/x86-64 (e.g. interaction with `ShadowGap`, the MidMem/prelink special case, or default mmap hint ranges)?
Contributor guide
Research direction
Start with the linked AddressSanitizer.cpp locations around the mapping offset and shadow instrumentation, then inspect the existing -asan-force-dynamic-shadow path and the ASAN_SHADOW_OFFSET_DYNAMIC and __asan_shadow_memory_dynamic_address entry points. Compare the fixed-offset and dynamic-shadow approaches, including the mentioned header and documentation changes. Done should mean an agreed upstream approach that avoids non-PIE collisions while preserving the default path for smaller binaries.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100