FEX-Emu / FEX-Emu/FEX

[SMT III Nocturne HD Remaster]: [Crash at GameAssembly.dll+0x9B50A8D — guest R11 arrives truncated to ~33 bits under ARM64EC]

Open
#5,893 2 comments 0 reactions 0 assignees View on GitHub
Game related ml-report
Dominant language
C++
Stars
8k
Forks
351
Avg merge
12h 31m
Merged PRs (30d)
102

Description

**What Game**

Shin Megami Tensei III Nocturne HD Remaster
https://store.steampowered.com/app/1413480/

**Describe the bug**

The game faults deterministically about 30-60 seconds after DXVK creates the D3D11 device, before reaching the title screen. It never renders a frame.

The fault is a write through guest `R11` at `GameAssembly.dll + 0x9B50A8D`:

```
Unhandled exception: page fault on write access to 0x00000000e124bf99
in 64-bit code (0x006fffe8fd0a8d).
=>0 0x006fffe8fd0a8d in gameassembly (+0x9b50a8d)
```

Guest instruction at that address, plus the three that follow:

```asm
49 89 8b c6 e4 95 a4 mov [r11 - 0x5B6A1B3A], rcx
4c 8b 1c 24 mov r11, [rsp]
48 83 c4 08 add rsp, 8
c3 ret
```

**The generated code for this block is correct — the input register is not.** From the ARM64 register dump at the fault:

```
Pc:00006fffe8fd0a8d
x0: 00006fffe1da0a70 <- guest RCX (stable module pointer, plausible)
x5: 000000013d026bce <- guest R11
x8: 00000000e1985094 <- computed store address
```

`x5 - 0x5B6A1B3A == x8` exactly. FEX materialised `R11` into `x5`, applied the displacement into `x8`, and stored `x0`. `R11` was already wrong when it arrived, so the defect is in whichever block produces it.

Guest `R11` across six runs:

```
1BAFF19B2 13ABC94AF 13CB8F3B3 13DA839CE 13D026BCE 13C8ED6D3
```

Every value is 33 bits with bit 32 set, and five of six share the leading `13`. All other registers are stable or vary plausibly, and the displacement is constant. A correct pointer here would be in the `0x6FFFE…` range. The shape suggests a 64-bit address computation performed at 32-bit width, or a lost zero-extension where a 32-bit result carried into bit 32 — not arbitrary corruption.

Two further details:

- **No assertion fires.** Built with `-DENABLE_ASSERTIONS=True` (enabling `IRValidation`), nothing trips before the fault. The IR appears structurally valid; the value in it is wrong.
- **The faulting code does not exist on disk.** RVA `0x9B50A8D` maps into `.sbss` (VMA `0x1831E4000`, size `0x13C01E4D`, image base `0x180000000`) — an uninitialised section marked executable, into which Denuvo generates code at runtime. Unity's own handler reports `RtlLookupFunctionEntry returned NULL`, consistent with that. There is no static reproduction path.

**To Reproduce**

1. Install the game on an AArch64 device and run it through Proton with the ARM64EC FEX backend (`libarm64ecfex.dll`).
2. Launch from Steam.
3. Wait ~30-60 seconds after DXVK reports D3D11 device creation.
4. The process faults at `GameAssembly.dll + 0x9B50A8D`. The module base moves between runs; the module-relative offset does not.

**Expected behavior**

The game reaches the title screen and runs.

**Screenshots and Video**

Not applicable — the crash happens before any frame is presented.

**System information:**

- OS: Armada (Fedora `bootc` based, aarch64)
- CPU/SoC:
- Video driver version:
- RootFS used: not applicable — the game runs via `libarm64ecfex.dll` inside a Proton prefix (`proton-cachyos-11.0-arm64`), not through the Linux binfmt path
- FEX version: reproduced with the Proton-shipped `libarm64ecfex.dll`, and with a self-built `FEX-2608-158-ga6e74cd` (`a6e74cdbe6f454376e0a55ac567f73f45a8888e3`), `RelWithDebInfo` + `-DENABLE_ASSERTIONS=True`
- Thunks Enabled: Yes — `GL`, `Vulkan`, `WaylandClient`, `asound`, `drm`. Also reproduces with the Vulkan thunk disabled.

**Additional context**

- Is this an x86 or x86-64 game: x86-64
- Does this reproduce on AArch64 with Radeon/Intel/Nvidia: Untested
- Is this a Vulkan game: Yes, via DXVK 3.0.2 translating D3D11
- If Yes, What is your Vulkan driver: Turnip, Mesa 26.2.0

Game details: Unity 2019.4.10f1, IL2CPP, Denuvo Anti-Tamper.

FEX configuration in effect (generated per-app by Armada, pointed to by `FEX_APP_CONFIG`) — note `Multiblock` is **0** in every run reported here:

```json
{
"HalfBarrierTSOEnabled": "1",
"MemcpySetTSOEnabled": "1",
"Multiblock": "0",
"TSOEnabled": "1",
"VectorTSOEnabled": "1",
"X87ReducedPrecision": "0"
}
```

**Ruled out** — no change in behaviour from any of:

- `TSOEnabled`, `ParanoidTSO`, `VectorTSOEnabled`, `MemcpySetTSOEnabled`, `HalfBarrierTSOEnabled` (all already 1)
- `SMCChecks=full`
- `Multiblock` (already 0 in all runs)
- `O0=1`
- `MaxInst` at 20, 500, 2000, 4000 and 5000
- Vulkan thunk disabled; `-force-gfx-direct`; fresh Wine prefix; Steam overlay disabled; `winegstreamer` disabled
- Both `proton-cachyos-11.0-arm64` and `GE-Proton11-6-aarch64`
- Code caching — no cache is written anywhere (`%LOCALAPPDATA%\fex-emu\` in the prefix and `~/.cache/fex-emu/` are both absent)

**Title-specific:** SMT V: Vengeance, also Denuvo-protected and from the same publisher, runs correctly on the same device and the same FEX build.

**Unexplained timing sensitivity.** Three conditions that substantially slow JIT compilation have coincided with the crash not occurring: a five-hour `FEX_DUMPIR` run that never reached the crash (173,115 blocks, 2.9 GB); five consecutive successful launches immediately afterwards at 20–25 fps with no FEX options set, a window that ended at reboot and has not recurred; and every subsequent run with the IR dumper active. I could not isolate a mechanism — FEXServer presence, `MaxInst` values and code caching were each tested and are not responsible. Noted in case it means something to someone who knows this code.

**Second fault site.** One run faulted instead at `gameassembly+0x12923DA`, reading address `0x4`, preceded by a `C000001D` illegal-instruction exception, with Unity's crash handler not running. The `+0x9B50A8D` site is far more common.

**Unaligned atomics.** A single startup logs roughly 73,600 `Handled unaligned atomic` fixups, each a signal-handler round trip. This dominates startup cost; possibly unrelated to the fault.

I have a working local ARM64EC build and can patch, rebuild and test the DLL, so I'm able to run diagnostics on request. Locating the block that produces the bad `R11` needs runtime IR dumping aimed at a specific guest address, since this code has no static form — any pointers on where a width or zero-extension defect would plausibly arise for runtime-generated code under ARM64EC would be welcome.

Contributor guide

Open the contributing guide

Research direction

Start with runtime IR dumping using FEX_DUMPIR, targeting the guest block that produces R11 before GameAssembly.dll+0x9B50A8D; the faulting code is generated at runtime and has no static reproduction path. Compare the generated IR and register-width handling under ARM64EC, then reproduce the game launch to verify that R11 is no longer truncated and the title screen is reached.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, linux
Domain
operating-systems, reverse-engineering
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.