NVIDIA / NVIDIA/open-gpu-kernel-modules

RTX 5090 (GB202): GNOME Wayland suspend never reaches S3 — gnome-shell stays runnable in nvidia.ko, nvidia-sleep.sh deadlocks (595.84 open, UseKernelSuspendNotifiers=0)

Open
#1,291 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
17.4k
Forks
1.9k
PR merge metrics
No merged PRs in 30d

Description

NVIDIA Open GPU Kernel Modules Version

595.84 (Ubuntu package nvidia-driver-595-open / nvidia-kernel-common-595 595.84-0ubuntu0.24.04.1)

NVRM: loading NVIDIA UNIX Open Kernel Module for x86_64 595.84

Proprietary driver comparison

RTX 5090 / GB202 requires the open kernel modules on this platform. The proprietary-only package is not a viable alternative test. Happy to retest a build NVIDIA specifies.

This hang is inside nvidia.ko (gvaspaceAlloc_IMPL / dmaAllocMapping_GM107 / nv_procfs_write_suspend), not in a userspace compositor bug alone.

Operating System and Version

Ubuntu 24.04.4 LTS (Noble)

Kernel Release

Linux redhead 7.0.0-28-generic #28~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC (Ubuntu HWE, stable, not -rc)

Hardware: GPU

GPU 0: NVIDIA GeForce RTX 5090 (UUID: GPU-d695c932-045f-c37d-e2b6-9bc983de4386) PCI 0000:01:00.0

  • Board: MSI MS-7D89, Intel i9-14900K, 188 GiB RAM
  • Display: LG 3440x1440 on NVIDIA HDMI (card2-HDMI-A-5)
  • Intel UHD 770 iGPU is present but unused for display
  • Session: GNOME 46 / Wayland (ubuntu-wayland)
Driver PM config
PreserveVideoMemoryAllocations: 1
UseKernelSuspendNotifiers: 0
EnableS0ixPowerManagement: 0
TemporaryFilePath: /var
mem_sleep: s2idle [deep]
nvidia-drm.modeset=1
nvidia-suspend.service / nvidia-resume.service: enabled

/var has ~748G free (VRAM dump space is not the limiter).

Describe the bug

On GNOME Wayland, systemctl suspend / the GNOME menu does not enter S3. The display goes black, fans and case lighting stay on, keyboard and power button do nothing. Hard power-off is required.

This is the userspace nvidia-sleep.sh path (UseKernelSuspendNotifiers=0), not the kernel PM-notifier / GSP-unload path in #1284.

Failed cycle (2026-08-12 22:53):

  1. nvidia-suspend.service runs nvidia-sleep.sh suspend (including chvt 63 + write to /proc/driver/nvidia/suspend). NVRM assertions fire in gmmu_walk.c / mmu_walk_sparse.c during that write.
  2. PM: suspend entry (deep)
  3. Userspace freeze fails after 20.000s: gnome-shell state:R, stack in NVIDIA (gvaspaceAlloc_IMPLdmaAllocMapping_GM107kbusMapFbAperture_GM107memMap_IMPL).
  4. Kernel retries s2idle; freeze fails again (2 tasks).
  5. Failed to put system to sleep. System resumed again: Device or resource busy
  6. nvidia-sleep.sh then blocks >122s in nv_procfs_write_suspend; kernel reports it is blocked on an rw-semaphore owned by gnome-shell.
  7. systemd SIGKILLs nvidia-sleep.sh. GPU reload fails:
NVRM: nvAssertOkFailedNoLog: ... NV_ERR_GENERIC ... gpuStateLoad(...) @ gpu_suspend.c:281
  1. Display stays off; machine stays fully powered. A later 07:01 retry starts from this already-broken GPU (Flip event timeout on head 0, Failed to allocate NVKMS memory for GEM object) and ends at PM: suspend entry (deep) with no further kernel logs until a hard reset the next morning.

Confirmed workaround (not a driver fix): SIGSTOP gnome-shell before nvidia-suspend.service, SIGCONT after nvidia-resume.service.

With that, the same machine:

  • freeze completes in 0.001s
  • enters ACPI S3 (Preparing to enter system sleep state S3 / Waking up from system sleep state S3)
  • power LED blinks
  • power-button press resumes to the lock screen
  • GNOME menu path and systemctl suspend both succeed (tested 2026-08-13)

So the GPU can suspend/resume on this board/BIOS/S3 path. The failure is gnome-shell remaining runnable inside NVIDIA ioctl/RM code across the freezer + procfs suspend handshake.

Bypassing systemd (rtcwake -m mem, no nvidia-sleep.sh) fails immediately, as documented:

NVRM: PreserveVideoMemoryAllocations module parameter is set. System Power Management attempted without driver procfs suspend interface.
nvidia 0000:01:00.0: PM: pci_pm_suspend(): nv_pmops_suspend returns -5
To Reproduce
  1. Ubuntu 24.04 + nvidia-driver-595-open 595.84 on RTX 5090.
  2. Enable NVreg_PreserveVideoMemoryAllocations=1 (required for GNOME Wayland).
  3. Leave UseKernelSuspendNotifiers=0 (Ubuntu default; nvidia-suspend.service enabled).
  4. Log into GNOME Wayland with the NVIDIA card driving the display.
  5. Suspend from the system menu or systemctl suspend.
  6. Observe: display off, fans on, no S3, no wake. Journal matches the freeze / nv_procfs_write_suspend deadlock above.

Precondition: do not SIGSTOP gnome-shell first.

Bug Incidence

Always (menu suspend and systemctl suspend) until gnome-shell is paused. After the pause workaround: 2/2 successful S3 + resume cycles, including the GNOME menu + power-button wake.

Related issues
  • #472 — PreserveVideoMemoryAllocations without procfs interface (same -5 if systemd units are skipped)
  • #1117 — Blackwell + Linux 7.0 s2idle hang (different path; we use deep / S3)
  • #1284 — RTX 5090 suspend Xid 120 with UseKernelSuspendNotifiers=1 on 610.x (we are on 595.84 userspace-sleep path)
  • Ubuntu already ships SYSTEMD_SLEEP_FREEZE_USER_SESSIONS=false in nvidia-kernel-common to avoid the Xorg/VT-switch freezer deadlock. The kernel freezer still hits this Wayland/gnome-shell case.
  • GNOME: VT switch in nvidia-sleep.sh is Xorg-oriented (gnome-settings-daemon#903)
What a real driver fix would look like

gnome-shell (and any Wayland compositor) issuing NVIDIA ioctls during nvidia-sleep.sh / freezer should be interruptible / freezer-aware. Today those threads stay R in RM (gvaspaceAlloc / BAR1 map) and then deadlock with nv_procfs_write_suspend. SIGSTOP is only a userspace bandage.

Attachments

See gist (journals, systemd workaround units, nvidia-bug-report.log):

https://gist.github.com/robertvandervoort/5488a63af729e7a1f474d61c28f49653

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the nvidia-sleep.sh suspend path and the nv_procfs_write_suspend call, then compare the reported gnome-shell stack with the related suspend behavior in #1284. Trace how the freezer interacts with the NVIDIA RM mappings and procfs suspend handshake. Done means suspend reaches S3 and resumes reliably without SIGSTOPing gnome-shell or leaving the GPU in a broken state.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, shell
Domain
computer-graphics, operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.