System-wide GPU/driver deadlock

NVIDIA Open GPU Kernel Modules Version

610.57.04

Please confirm this issue does not happen with the proprietary driver (of the same version). This issue tracker is only for bugs specific to the open kernel driver.

• I confirm that this does not happen with the proprietary driver package.

Operating System and Version

Alma Linux 10.2

Kernel Release

6.12.0-211.40.1.el10_2.x86_64

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.

• I am running on a stable kernel release.

Hardware: GPU

8x L20

Describe the bug

(This an AI-generated description, but it is accurate.)

Under concurrent CUDA context initialization on a multi-GPU system, if an RM system-memory descriptor allocation hits NV_ERR_NO_MEMORY, the driver enters a pathological cleanup path in kernel-open/nvidia/nv-vm.c: nv_set_memory_type() that calls set_memory_wb() on physical page 0 for every unallocated slot of the descriptor. This produces a flood of kernel x86/PAT: freeing invalid memtype [mem 0x0-0xfff] messages and an O(num_pages) TLB-flushing loop that runs while holding the RM API global write lock (rmapiLockAcquire in serverAllocResource). All other threads block on that lock forever — the whole GPU stack (and typically the box) wedges and requires a hard reset.

To Reproduce

// repro.cu  —  nvcc -o repro repro.cu
#include <cuda_runtime.h>
#include <cstdio>
#include <cstdlib>
int main(int argc, char** argv) {
    int dev = (argc > 1) ? atoi(argv[1]) : 0;
    cudaSetDevice(dev);
    cudaError_t e = cudaFree(0);      // force primary context creation
    if (e != cudaSuccess) { printf("FAILED: %s\n", cudaGetErrorString(e)); return 1; }
    printf("OK\n");
    return 0;
}

# 64 concurrent context inits spread across 8 GPUs
for i in $(seq 0 63); do ./repro $((i % 8)) & done; wait

Bug Incidence

Always

nvidia-bug-report.log.gz

nvidia-bug-report.log

More Info

Loading the driver with uvm_hmm_disable=1 reliably avoids the failure. So does downgrading to 595.58.03.

Possibly adjacent to #1223, but clearly distinct.

Start with kernel-open/nvidia/nv-vm.c and the nv_set_memory_type() path described in the report; use repro.cu and the 64-process shell loop to reproduce the failure. Compare behavior with uvm_hmm_disable=1 and driver 595.58.03, then use the kernel messages and lockup behavior to verify the issue is resolved without requiring a hard reset.