CUDA "device not ready" during multi-GPU model load: dxgkio_make_resident fails with ENOMEM while nvidia-smi shows ~2 GB free (related: #41017, #40580)

Nobody has claimed this yet.

# Body

## Summary

While loading a second model (a speculative-decoding draft) immediately after a large model load in a 2-GPU tensor-parallel LLM server on WSL2, the application deterministically fails with:

```
RuntimeError: CUDA driver error: device not ready
```

This happened **5 out of 5 attempts**, always at the first GPU allocation performed for the second model. Each userspace failure coincides — to the second — with exactly **two** kernel-side paging failures (one per GPU):

```
misc dxg: dxgk: dxgkio_make_resident: Ioctl failed: -12 # ENOMEM
```

At that moment `nvidia-smi` reports roughly **2 GB free per GPU** (about 9.6 / 12.2 GiB used), so the residency rejection happens well below device capacity. The same day, an unrelated workload also produced a different failure signature resembling #41017:

```
misc dxg: dxgk: dxgvmb_send_create_allocation: send_create_allocation failed ffffffb5 # -75 EOVERFLOW
misc dxg: dxgk: dxgkio_create_allocation: Ioctl failed: -75
misc dxg: dxgk: dxgvmb_send_sync_msg: wait_for_completion failed: fffffe00
```

I'm filing this primarily as a **breadcrumb for others** who hit the extremely misleading "device not ready" error from CUDA on WSL2: in our case it was the dxg paging layer refusing new residency (ENOMEM), not a device/driver readiness problem. Issue #41017 (same userspace error string, same kernel version) was auto-closed for missing logs; this report includes the kernel-side evidence.

## Environment

| Item | Value |
|---|---|
| Windows | 10.0.26200.8875 |
| WSL | 2.7.10.0 (store) |
| WSL kernel | 6.18.33.2-microsoft-standard-WSL2 |
| WSLg / Direct3D / DXCore | 1.0.73.2 / 1.611.1 / 10.0.26100.1 |
| GPU | 2× NVIDIA GeForce RTX 5070 (12227 MiB each) |
| NVIDIA driver (Windows) | 596.36 (R595 branch) |
| CUDA (via WSL libcuda) | 13.2 |
| Workload | Aphrodite Engine 0.22.0 (vLLM-family LLM server), PyTorch 2.x, tensor-parallel across both GPUs, `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` |

GPU0 also drives the Windows desktop (~790 MB baseline usage); GPU1 is headless.

## Reproduction (our real-world case)

1. Start a 2-process tensor-parallel LLM server; each worker loads ~9.56 GiB of model weights onto its GPU (this load **always succeeds** — 4/4 boots that day).
2. Immediately after the main model load, each worker constructs a second, small model (~250 MB speculative-decoding draft) on the same GPU.
3. The very first CUDA allocation for the second model fails with `RuntimeError: CUDA driver error: device not ready`. With one allocation site patched to CPU, the failure simply moves to the next GPU allocation (`torch.empty` for a small weight tensor), so it is not tied to any specific op or allocation size.

Correlation across all 5 attempts (timestamps local, 2026-07-26):

| Userspace failure (app log) | dmesg |
|---|---|
| 09:21 | 09:21:08 `dxgkio_make_resident: Ioctl failed: -12` ×2 |
| 09:28 | 09:28:03 same ×2 |
| 09:35 | 09:35:05 same ×2 |
| 09:44 | 09:44:36 same ×2 |
| 09:49 | 09:49:34 same ×2 |

No other dxg errors occur during these boots apart from the usual benign `dxgkio_query_resource_info_nt: invalid fd` noise, which also appears on fully successful boots.

## Questions / expected behavior

1. **Error surfacing**: a residency/paging failure reaching the app as `cudaErrorNotReady` ("device not ready") is very hard to diagnose. If dxgkrnl/paging rejects an allocation, surfacing it as an out-of-memory condition would save users a lot of time.
2. **Budget accounting**: `dxgkio_make_resident` returns ENOMEM while `nvidia-smi` shows ~2 GB free per device. Is there a per-process WDDM residency budget on WSL2 that sits meaningfully below device capacity, and is it queryable (or tunable) from the Linux side? (Native Windows apps can query `QueryVideoMemoryInfo`; nothing equivalent seems exposed in WSL.)
3. The `-75 EOVERFLOW` + `wait_for_completion failed: fffffe00` signature (observed the same day under a different workload, and matching what #41017 / #40580 describe around `dxgvmbus.c` field-spanning writes) appears to be a distinct bug from the ENOMEM path above. It may be worth keeping them separate when triaging.

Happy to provide full dmesg, app-side tracebacks, or run diagnostics on request — the failure is deterministic in our setup, reproducible within ~90 seconds per attempt.

Start by reviewing the reported dxgkio_make_resident ENOMEM messages in dmesg and the dxgvmbus.c context mentioned in the report. Reproduce the two-GPU model-load sequence and compare it with nvidia-smi usage. Done means determining whether a WSL2 residency budget or error-mapping issue explains the failure, while keeping the separate EOVERFLOW signature distinct.

Written by the indexing model from the issue text.