NVIDIA / NVIDIA/open-gpu-kernel-modules

RTX 4090 (AD102): dmaAllocMapping_GM107 VA space exhaustion → RC watchdog GPU lock after 7 days uptime (nvidia-open 595.58.03)

Open
#1,140 18 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
17.4k
Forks
1.9k
PR merge metrics
No merged PRs in 30d

Description

RTX 4090 (AD102): dmaAllocMapping_GM107 VA space exhaustion → RC watchdog GPU lock after 7 days uptime (nvidia-open 595.58.03)

System Information

Component Value
GPU NVIDIA GeForce RTX 4090 (AD102, rev a1) — MSI (subsystem 1462:5104)
VBIOS 95.02.18.80.74
Driver nvidia-open 595.58.03-5 (Arch nvidia-open package)
Kernel 6.19.13-arch1-1 (x86_64)
CPU AMD Ryzen 9 7950X
RAM 64 GB
OS Arch Linux
Display server Hyprland 0.54.3 (Wayland)
PCIe Gen 1 x16 (current link speed per nvidia-smi)
Module options nvidia_drm modeset=1
Resizable BAR N/A (BAR1 256MB, BAR3 32MB)

Bug Description

After approximately 7 days of uptime under normal desktop use (Wayland compositor, browser, terminals), the nvidia-open driver began failing DMA VA space allocations, eventually leading to a complete GPU lockup that required a hard reboot to recover.

Timeline

  1. May 3 12:38 — System boot. nvidia-open 595.58.03 loads normally.
  2. May 10 13:42 — First burst of 19× dmaAllocMapping_GM107: can't alloc VA space for mapping errors in rapid succession.
  3. May 10 14:44–14:50 — Three more bursts of DMA allocation failures (22, 15, 17, 19 occurrences).
  4. May 10 17:24 — Hyprland crashes (coredump in CCompositor::initServer). RC watchdog fires:
    NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
    NVRM: nvAssertFailedNoLog: Assertion failed: GPPut < WATCHDOG_GPFIFO_ENTRIES @ kernel_rc_watchdog.c:1549
    
  5. May 10 17:24–17:35 — SDDM attempts to restart Hyprland 3 times; each attempt results in the same crash at CCompositor::initServer. Display server fails to start on vt 2.
  6. May 10 17:36 — Manual attempts to reload the nvidia modules via modprobe -r nvidia_drm nvidia_modeset nvidia_uvm nvidia fail — GPU is hung.
  7. May 10 17:36–17:38 — RC watchdog continues firing every ~8 seconds until system reboot. The GPU never recovers.
  8. May 10 17:38 — System rebooted. GPU functions normally on the new boot, no errors present.

Logs

DMA allocation failures (first burst):

May 10 13:42:15 kernel: NVRM: dmaAllocMapping_GM107: can't alloc VA space for mapping.
[repeated 19 times]

RC watchdog (repeating every ~8s from 17:36 until reboot):

May 10 17:36:04 kernel: NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
May 10 17:36:05 kernel: NVRM: nvAssertFailedNoLog: Assertion failed: GPPut < WATCHDOG_GPFIFO_ENTRIES @ kernel_rc_watchdog.c:1549

Hyprland coredump (all 3 restart attempts identical):

Process (Hyprland) of user 1000 dumped core.
#10 /usr/bin/Hyprland + 0x15d0af
#11 CCompositor::initServer (/usr/bin/Hyprland + 0x2ac4c0)
#12 main (/usr/bin/Hyprland + 0x206c53)

PCI BAR layout:

BAR0: 0xfb000000–0xfbffffff  (16 MB, 32-bit non-prefetchable)
BAR1: 0xd0000000–0xdfffffff  (256 MB, 64-bit prefetchable)
BAR3: 0xe0000000–0xe1ffffff  (32 MB, 64-bit prefetchable)

Notes

  • No Xid errors were logged.
  • The system has 24 GB VRAM. Workload at the time of failure was a normal desktop session — not a heavy VRAM consumer.
  • The proprietary driver was not tested.
  • SSH remained functional after display loss, suggesting the host CPU/kernel was not frozen — only the GPU was locked.
  • A separate set of AMD MCE hardware errors (bogus, all-0xffffffffffffffff MC31_STATUS values from an unrelated kernel regression) were present throughout the boot but are not believed to be related to the GPU failure.

Related Issues

  • #1078 — Same driver (595.58.03), same dmaAllocMapping_GM107 errors on RTX 4060 Ti
  • #1132 — Same symptom chain (dmaAllocMapping → krcWatchdog GPU lock) on RTX 5070 / 595.71.05
  • #1134 — BAR mapping VA space exhaustion on RTX 3090 / 595.71.05
  • #971 — GPU hang after extended uptime (~66 days) on B200 / 570.133.20

This appears to be part of a regression in the 580–595 open driver series affecting DMA VA space management across multiple GPU generations.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the DMA allocation failures and RC watchdog logs, then read kernel_rc_watchdog.c around line 1549 and compare related issues #1078, #1132, and #1134. Done would require identifying the DMA VA exhaustion cause and validating a driver-level fix against the long-uptime failure sequence.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, linux
Domain
operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.