CachyOS / CachyOS/linux-cachyos

[BUG] Navi 10 (RX 5700 XT): kwin_wayland gfxhub page fault at GTT base+0x1000 on multi-monitor window drag -> gfx_0.0.0 ring reset fails -> BACO reset loop (devcoredump captured)

Open
#952 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
4.5k
Forks
160
Avg merge
2d 12h
Merged PRs (30d)
1

Description

### Pre-flight checklist

- [x] I have searched existing issues and this is not a duplicate. (Closest match is #928 — same fault→ring-reset-failed→full-reset pattern, but different silicon (Granite Ridge iGPU, GC 10.3.6) and a different UTCL2 client. Filing separately as requested there; happy to have this merged if you consider it the same root cause.)
- [x] I have read the Contributing Guidelines.
- [x] I have verified the issue is reproducible with the latest available CachyOS kernel (`linux-cachyos 7.1.4-1` from `cachyos-v3`, which is current).
- [ ] **I have NOT tried to reproduce on Arch Linux's `linux` kernel.** I'm flagging this openly rather than ticking the box. See the note below — I have strong indirect evidence this is not CachyOS-specific, but no vanilla test of my own.

### Upstream / vanilla kernel check

**I have not tested with a vanilla/upstream kernel.**

I want to be straight about that, and also explain why I still think this is worth your time rather than a mis-filed distro bug:

The identical fault exists in **[KDE bug 453147](https://bugs.kde.org/show_bug.cgi?id=453147)**, filed in 2022 against **the same GPU (RX 5700 XT)** on stock Arch with Mesa 22.0.2. That report contains the same faulting thread string (`process kwin_wayland pid … thread kwin_wayla:cs0 pid …`), the same `ring gfx_0.0.0 timeout` → failed ring reset → BACO escalation, and the same `[drm] *ERROR* Failed to initialize parser -125!` on resume. So the signature predates this kernel branch by four years and has survived many Mesa and kernel releases on a non-CachyOS kernel.

If you'd still rather I test vanilla before you spend time on it, say so and I'll close this — but I'd rather leave the data on the record than lose it, because the artifact below is hard to capture (see "How the coredump was obtained").

### Kernel variant

`linux-cachyos` (EEVDF, Clang)

### System information

inxi -Faz (System / Machine / CPU / Graphics; serials and UUIDs redacted)

```text
System:
Kernel: 7.1.4-1-cachyos arch: x86_64 bits: 64 compiler: clang v: 22.1.8 clocksource: tsc
avail: hpet,acpi_pm parameters: quiet splash rw rootflags=subvol=/@
root=UUID= systemd.tpm2_wait=no
Desktop: KDE Plasma v: 6.7.3 tk: Qt v: N/A info: frameworks v: 6.28.0 wm: kwin_wayland vt: 2
dm: N/A Distro: CachyOS base: Arch Linux
Machine:
Type: Desktop System: Gigabyte product: X470 AORUS ULTRA GAMING v: N/A
Mobo: Gigabyte model: X470 AORUS ULTRA GAMING-CF
Firmware: UEFI vendor: American Megatrends LLC. v: F62 date: 01/24/2022
CPU:
Info: model: AMD Ryzen 7 2700X bits: 64 type: MT MCP arch: Zen+ gen: 1+ level: v3
built: 2018-21 process: GF 12nm family: 0x17 (23) model-id: 8 stepping: 2 microcode: 0x800820D
Topology: cpus: 1x dies: 1 cores: 8 threads: 16 tpc: 2 smt: enabled
Speed (MHz): min/max: 2200/4350 boost: enabled scaling: driver: acpi-cpufreq governor: schedutil
Graphics:
Device-1: Advanced Micro Devices [AMD/ATI] Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT]
vendor: Gigabyte driver: amdgpu v: kernel arch: RDNA-1 code: Navi-1x process: TSMC n7 (7nm)
built: 2019-20 ports: active: DP-1,DP-2 empty: DP-3,HDMI-A-1
bus-ID: 0b:00.0 chip-ID: 1002:731f class-ID: 0300
Display: wayland server: Xwayland v: 24.1.13 compositor: kwin_wayland driver: gpu: amdgpu
d-rect: 3840x1080 display-ID: 0
Monitor-1: DP-1 pos: primary,left model: LG (GoldStar) ULTRAGEAR built: 2021
res: mode: 1920x1080 hz: 144 scale: 100% (1) diag: 685mm (27")
Monitor-2: DP-2 pos: right model: LG (GoldStar) ULTRAGEAR built: 2021
res: mode: 1920x1080 hz: 144 scale: 100% (1) diag: 685mm (27")
API: OpenGL v: 4.6 vendor: amd mesa v: 26.1.5-arch3.1 direct-render: yes
renderer: AMD Radeon RX 5700 XT (radeonsi navi10 ACO DRM 3.64 7.1.4-1-cachyos)
device-ID: 1002:731f memory: 7.81 GiB unified: no
API: Vulkan v: 1.4.350 device: 0 type: discrete-gpu name: AMD Radeon RX 5700 XT
(RADV NAVI10) driver: mesa radv v: 26.1.5-arch3.1 device-ID: 1002:731f
```

Additional, from the device coredump:

```text
VBIOS : GV-R57XTGAMING OC-8GD/F81/0A07, ver 017.001.000.049.000000, 2020/02/26
HWIP GC : v10.1.10 DCE: v2.0.2 MMHUB: v2.0.0 ATHUB: v2.0.0 NBIO: v2.3.0
GFX firmware : ME 0x65 PFP 0xa0 CE 0x26 RLC 0x80 MEC 0x9d MEC2 0x9d
real vram : 8573157376
visible vram : 268435456 <- 256 MB BAR, not a large-BAR system
gtt size : 8338014208
```

RAM: 16 GiB. No SCX scheduler active (`sched_ext: disabled`). No GPU overclock or undervolt: `power_dpm_force_performance_level=auto`, power cap 190 W = `cap_max`, OverDrive bit clear. Only a custom fan curve (via LACT, junction-targeted) is applied.

### Bug description

On an RX 5700 XT (Navi 10) under KDE Plasma 6.7.3 / Wayland with **two displays**, the gfx ring repeatedly faults with a GPUVM permission fault. The faulting process is **always `kwin_wayland`**, on its radeonsi command-submission thread `kwin_wayla:cs0` — not the foreground application. The fault address is the GART/GTT aperture base plus exactly one page.

There are arguably **two separate bugs** here:

**1. The fault itself.** A CS submitted by radeonsi on behalf of the compositor reads a GTT page where the page-table walk *succeeds* but the permissions are wrong:

```
PERMISSION_FAULTS: 0x3 MAPPING_ERROR: 0x0 WALKER_ERROR: 0x0 RW: 0x0
```

A successful walk with bad permissions on a *read*, at a suspiciously round aperture-relative address, does not look like a wild pointer. It looks like a CS still referencing a buffer whose GTT mapping has been torn down.

**2. Recovery is unrecoverable — this is the part I'd most like looked at.** Per-ring reset **always** fails, the driver escalates to BACO, VRAM is lost, re-init fails with `-ECANCELED`, and the machine then enters a self-sustaining reset loop roughly every 5 seconds. The session cannot be recovered and the physical reset button is required. A page fault caused by a userspace GL client should not require hardware intervention.

### Steps to reproduce

1. Two displays, both 1920x1080@144 on DisplayPort, side by side, no scaling, VRR off. KDE Plasma Wayland (KWin on its OpenGL/radeonsi backend).
2. Put a GPU-accelerated application under sustained load on the left monitor (in my case Path of Exile via Proton, but the renderer is irrelevant — see below).
3. Open a second GPU-accelerated window (in my case Path of Building, a Flatpak Wayland client).
4. **Drag that second window from one monitor to the other.**

The most recent occurrence happened *at the instant* the window crossed onto the second monitor. Every prior occurrence had at least one window open on the second monitor. Sessions with the second monitor left empty run substantially longer before faulting.

Time-to-fault is otherwise unpatterned: 44 s, 49 s, 64 s, ~9 min, 22 min, 2 h 34 m. So this reproduces within minutes given the window-drag trigger, but I can't yet promise a deterministic one-shot repro.

### Expected behavior

Moving a window between outputs should not fault the gfx ring. Failing that, a GPUVM fault caused by a userspace GL client should be recovered by a per-ring or per-VM reset, killing at most the offending context — not lose VRAM and enter a reset loop that requires the reset button.

### Actual behavior

`[gfxhub] page fault` → `ring gfx_0.0.0 timeout` → `Ring gfx_0.0.0 reset failed` → `BACO reset` → `VRAM is lost due to GPU reset!` → `Failed to initialize parser -125` → loop repeats indefinitely → hard reset required.

### Logs / stack traces

Fault and escalation sequence (hostname stripped):

```text
15:34:21 kernel: amdgpu 0000:0b:00.0: [gfxhub] page fault (src_id:0 ring:24 vmid:5 pasid:12)
15:34:21 kernel: amdgpu 0000:0b:00.0: Process kwin_wayland pid 1664 thread kwin_wayla:cs0 pid 1673
15:34:21 kernel: amdgpu 0000:0b:00.0: in page starting at address 0x0000800000001000 from client 0x1b (UTCL2)
15:34:21 kernel: amdgpu 0000:0b:00.0: GCVM_L2_PROTECTION_FAULT_STATUS:0x00500C30
15:34:21 kernel: amdgpu 0000:0b:00.0: Faulty UTCL2 client ID: CPG (0x6)
15:34:21 kernel: amdgpu 0000:0b:00.0: MORE_FAULTS: 0x0
15:34:21 kernel: amdgpu 0000:0b:00.0: WALKER_ERROR: 0x0
15:34:21 kernel: amdgpu 0000:0b:00.0: PERMISSION_FAULTS: 0x3
15:34:21 kernel: amdgpu 0000:0b:00.0: MAPPING_ERROR: 0x0
15:34:21 kernel: amdgpu 0000:0b:00.0: RW: 0x0
15:34:23 kernel: amdgpu 0000:0b:00.0: Dumping IP State
15:34:23 kernel: amdgpu 0000:0b:00.0: Dumping IP State Completed
15:34:23 kernel: amdgpu 0000:0b:00.0: [drm] AMDGPU device coredump file has been created
15:34:23 kernel: amdgpu 0000:0b:00.0: ring gfx_0.0.0 timeout, signaled seq=839852, emitted seq=839854
15:34:23 kernel: amdgpu 0000:0b:00.0: Process kwin_wayland pid 1664 thread kwin_wayla:cs0 pid 1673
15:34:23 kernel: amdgpu 0000:0b:00.0: Starting gfx_0.0.0 ring reset
15:34:23 kernel: amdgpu 0000:0b:00.0: Ring gfx_0.0.0 reset failed
15:34:23 kernel: amdgpu 0000:0b:00.0: GPU reset begin!. Source: 1
15:34:23 kernel: amdgpu 0000:0b:00.0: BACO reset
15:34:25 kernel: amdgpu 0000:0b:00.0: GPU reset succeeded, trying to resume
15:34:25 kernel: amdgpu 0000:0b:00.0: [drm] PCIE GART of 512M enabled (table at 0x0000008000800000).
15:34:25 kernel: amdgpu 0000:0b:00.0: VRAM is lost due to GPU reset!
15:34:25 kernel: amdgpu 0000:0b:00.0: PSP is resuming...
15:34:26 kernel: amdgpu 0000:0b:00.0: SMU is resuming...
15:34:26 kernel: amdgpu 0000:0b:00.0: SMU is resumed successfully!
15:34:26 kernel: amdgpu 0000:0b:00.0: kiq ring mec 2 pipe 1 q 0
15:34:26 kernel: amdgpu 0000:0b:00.0: [drm] *ERROR* Failed to initialize parser -125!
15:34:26 kernel: amdgpu 0000:0b:00.0: GPU reset(1) succeeded!
15:34:28 kernel: amdgpu 0000:0b:00.0: Dumping IP State
15:34:28 kernel: amdgpu 0000:0b:00.0: [drm] AMDGPU device coredump file has been created
15:34:28 kernel: amdgpu 0000:0b:00.0: ring gfx_0.0.0 timeout, signaled seq=839856, emitted seq=839858
15:34:28 kernel: amdgpu 0000:0b:00.0: Starting gfx_0.0.0 ring reset
15:34:28 kernel: amdgpu 0000:0b:00.0: Ring gfx_0.0.0 reset failed
15:34:28 kernel: amdgpu 0000:0b:00.0: GPU reset begin!. Source: 1
15:34:28 kernel: amdgpu 0000:0b:00.0: BACO reset
... repeats until the reset button is pressed
```

Hang-state registers from the coredump at fault time:

```text
Ring timed out details
IP Type: 0 Ring Name: gfx_0.0.0
[gfxhub] Page fault observed
Faulty page starting at address: 0x0000800000001000
Protection fault status register: 0x500c30

mmGRBM_STATUS 0xa0003028
mmGRBM_STATUS2 0x50000008
mmGRBM_STATUS3 0x00000000
mmCP_STALLED_STAT1 0x00000c00
mmCP_BUSY_STAT 0x00008000

Ring buffer information
ring name: gfx_0.0.0
Rptr: 0x762 Wptr: 0xb79aa00 RB mask: 7ff Ring size in dwords: 2048
```

### Fault signature across occurrences

All with `PERMISSION_FAULTS: 0x3`, `MAPPING_ERROR: 0x0`, `WALKER_ERROR: 0x0`, and faulting process `kwin_wayland` / thread `kwin_wayla:cs0`:

| # | vmid:pasid | Fault address | STATUS | Client |
|---|---|---|---|---|
| 1 | 6:13 | `0x0000800001fbd000` | `0x00600C30` | CPG (0x6) |
| 2 | 5:12 | `0x0000800000001000` | `0x00500C30` | CPG (0x6) |
| 3 | 5:12 | `0x0000800010000000` | `0x0050153A` | SQC (data) (0xa) |
| 4 | 5:12 | `0x0000800000001000` | `0x00500C30` | CPG (0x6) |

GTT aperture base is `0x0000800000000000`, so `0x0000800000001000` is base + exactly one page. Earlier occurrences produced similarly aperture-relative addresses (`0x800002017000`, `0x800010200000`).

Worth noting: #928's fault address on entirely different silicon is **also aperture base + 0x1000** (`0x4380001000`), with client TCP (0x8) rather than CPG.

### How the coredump was obtained

Relevant because it's why this report has an artifact at all. `devcoredump` entries expire after ~5 minutes and are lost outright on a hard reset — and since this bug requires the reset button, the dump was being destroyed every time. I ended up installing a udev rule to copy it off the moment it appears:

```
# /etc/udev/rules.d/99-amdgpu-devcoredump.rules
SUBSYSTEM=="devcoredump", ACTION=="add", RUN+="/usr/local/bin/save-devcoredump.sh %k"
```

The script copies `/sys/class/devcoredump/$1/data` to `/var/log/` and `sync`s it. On this occurrence it captured the dump **7 seconds before the reboot**. Offering it in case it's useful to other people chasing reset-loop bugs — anyone hitting this class of issue is probably losing their dumps the same way.

**The full 5.7 MB / 270211-line `AMDGPU Device Coredump` (version 1) is available on request** — SOC info, HW IP discovery, IP firmware versions, VBIOS, ring-timeout details, the complete IP register dump, and gfx_0.0.0 ring contents. Tell me how you'd like it (gist, paste.cachyos.org, attachment) and I'll upload. I also have a 5-second-interval junction/memory/edge/fan/power/sclk/mclk trace covering the whole crash session.

### Additional system information

**Eliminated by test, so you don't have to ask:**

| Hypothesis | Evidence against |
|---|---|
| Thermal | Fault occurred at **59 °C junction / 62 °C memory / 38–54 W** against a 190 W cap. Session peak was only 67 °C / 64 °C / 117 W. From a 5 s-interval trace across the whole session. |
| VRAM defect | `memtest_vulkan` clean — ~140k iterations, 401 GB/s, zero errors |
| mclk switching | mclk was **pinned at 875 MHz for all 273 samples** of the crash session — zero transitions. Both displays are the same refresh rate, so no mixed-refresh mclk churn either. |
| PCIe / link | Zero AER events, zero `Bad TLP`, no link retrain, no MCE, no EDAC errors |
| Power / OC | Fully stock, see system info above |
| Motherboard firmware | Flashed F2 → F62 (latest) mid-investigation. Fault signature byte-identical before and after. |
| System RAM | Zero OOM kills, 11 GiB of 15 GiB available at fault time |
| Foreground app's renderer | Faults with the app on Vulkan/RADV **and** on DX11/DXVK. The faulting CS belongs to the compositor either way. |
| `amdgpu.vm_update_mode` | Faults with it unset (`-1`) and with `3`. (`3` is invalid on this 256 MB BAR card and warns at `amdgpu_vm.c:2633`; it has been removed.) |

**Currently testing as a workaround:** `KWIN_DRM_FORCE_GL_FINISH_PRESENT=1`, on the theory that serialising the compositor's command stream against presentation closes the window in which a buffer can be released while a CS referencing it is still in flight. I'll report back whether it holds.

### Questions

1. Is per-ring reset (`Starting gfx_0.0.0 ring reset`) expected to work on GC 10.1.10? It fails 100% of the time here, which is what forces BACO and the VRAM loss.
2. After `VRAM is lost due to GPU reset!`, is the reset loop expected — or is the failure to re-init (`Failed to initialize parser -125` / `-ECANCELED`) the thing that makes it unrecoverable?
3. Is there instrumentation you'd like enabled to identify which BO is at `GTT base + 0x1000` at fault time? I'm happy to run debug patches or a kernel with extra logging; the fault reproduces within minutes given the window-drag trigger.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the fault with Arch's vanilla linux kernel, then inspect the amdgpu devcoredump and the gfxhub page-fault, gfx_0.0.0 ring-reset, and BACO-reset log sequence. Compare the evidence with issue #928 and KDE bug 453147. Done means identifying the fault or recovery cause and defining a reproducible fix that avoids the unrecoverable reset loop.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux
Domain
computer-graphics, operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.