GB10 (DGX Spark): silent whole-host wedge after NVRM: NV_ERR_NO_MEMORY (_memdescAllocInternal) under sustained unified-memory allocation pressure — no OOM kill, no panic, empty pstore, power cycle required

NVIDIA Open GPU Kernel Modules Version

595.84

Please confirm this issue does not happen with the proprietary driver (of the same version). This issue tracker is only for bugs specific to the open kernel driver.

• I confirm that this does not happen with the proprietary driver package.

Operating System and Version

Ubuntu 24.04.5 LTS (DGX OS flavour), aarch64

Kernel Release

Linux 6.17.0-1032-nvidia (aarch64)

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.

• I am running on a stable kernel release.

Hardware: GPU

GPU 0: NVIDIA GB10 (UUID: GPU-1e6e8ddd-283c-2fcf-cd6f-c00b19eb3090)

Describe the bug

Under sustained unified-memory allocation pressure from a userspace workload (vLLM
serving NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 per NVIDIA's own DGX Spark deployment
guide: ModelOpt NVFP4 + Marlin GEMM + Mamba-2 hybrid + baked-in MTP speculative
decoding, async-scheduling, chunked prefill, fp8 KV cache), a single long-prefill
burst (~40k tokens, one sequential request) makes the driver log

NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY]
(0x00000051) returned from _memdescAllocInternal(pMemDesc) @ mem_desc.c:1359

and then the ENTIRE HOST wedges silently: no kernel OOM kill (10-12 GiB RAM still
free), no panic, /sys/fs/pstore/ is empty, the journal stops abruptly right after the
NVRM lines. SSH/ICMP stop responding; the machine requires a manual power cycle.
Reproduced 3 times across driver 580.173.02 and 595.84, with and without
PYTORCH_ALLOC_CONF=expandable_segments:True (the mitigation from vLLM issue #55569,
same signature on GLM-NVFP4, does not hold for this NemotronH pipeline). Lighter
inference load (5k-token prompts) is stable.

An allocation failure must fail the userspace allocation (CUDA OOM error), never
silently wedge the host; at minimum a panic with a pstore/ramoops dump is needed as
a debug surface.

To Reproduce

1. On a DGX Spark (GB10, 121 GiB unified memory, Ubuntu 24.04.5 DGX OS, driver 595.84,
   Secure Boot on), serve nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 with vLLM
   cu130-nightly per NVIDIA's "Nemotron 3 Super - DGX Spark Deployment Guide"
   (vLLM section): gpu-memory-utilization 0.85 (NOTE: the guide value 0.90 does NOT
   reach a serving state on GB10 - see "More Info"), kv-cache fp8, moe-backend marlin,
   async-scheduling, chunked prefill, spec-decode MTP.
2. Startup completes cleanly (weights 75 GiB, KV pool ~32 GiB, autotuning done, API OK).
3. Light load is fine: 8 sequential ~5k-token prompts -> 8/8 HTTP 200.
4. Send ONE ~40k-token prompt (sequential, no concurrency): prefill runs for minutes,
   then NVRM NV_ERR_NO_MEMORY lines appear and the host wedges silently (no OOM kill,
   no panic, empty pstore) -> power cycle required.

Bug Incidence

Always

nvidia-bug-report.log.gz

Report generated after a reboot following the last wedge (system running normally).
The crash evidence is in the description: the journal of the crashed boot ends with
the quoted NVRM NV_ERR_NO_MEMORY lines from _memdescAllocInternal; no pstore entry
exists (part of the bug).

nvidia-bug-report.log.gz.gz

More Info

Additional data points from the same host (2026-09-12):

• gpu_memory_utilization 0.90 (the deployment guide value) OOM-killed the host during
  engine startup in 3/3 runs - system-wide kernel OOM kills (the vLLM container
  processes AND unrelated daemons: firewalld, sshd user managers, unbound, exporters),
  no panic, no pstore. The serving state is only reachable at 0.85. This matches vLLM
  issue #46307 (gpu_memory_utilization not respected on GB10 during startup profiling).
• All crashes occurred with no desktop/GDM session active (headless server).
• vLLM-side thread collecting the same signature (GLM-NVFP4, 2x Spark TP):
  vllm-project/vllm#55569.

Start at _memdescAllocInternal in mem_desc.c around line 1359, then reproduce the sustained unified-memory workload on the GB10 using the listed vLLM configuration and inspect the NVRM and journal output. Done means an allocation failure returns a userspace CUDA OOM error instead of wedging the host, with useful crash evidence such as a pstore or ramoops dump.