Comfy-Org / Comfy-Org/ComfyUI

BSOD in dxgmms2.sys (VidMm page-in use-after-free) on 6 GB RTX 3050 since v0.35.0 / comfy-aimdo 0.5.3 dynamic VRAM loading

Open
#16,246 5 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

## Summary

Since updating from **v0.34.6 to v0.35.0 / v0.35.1** (which bumped `comfy-aimdo` 0.4.15 → 0.5.3), my Windows 11 machine has hit **4 kernel BSODs in one day**, all with the identical signature inside the Windows video memory manager (`dxgmms2.sys`) while ComfyUI was staging models through the new dynamic VRAM loader on a 6 GB GPU. Two of the crashes happened at the **exact same log line** of the same workflow. With v0.34.6 the day before, the same machine ran 116 prompts over 12 hours without a single problem.

I'm aware a user-mode app "shouldn't" be able to bluescreen Windows — the fault is technically in `dxgmms2.sys` / `nvlddmkm.sys` — but the trigger is very clearly the allocation pattern of the 0.35 dynamic loader, and the startup message says `--disable-dynamic-vram` "will be removed soon" and asks for detailed reports, so here is one with a full kernel dump analysis.

## Environment

| | |
|---|---|
| ComfyUI | v0.35.0 (crash #1–#3) and v0.35.1 (crash #4), via ComfyUI Desktop 0.4.x auto-update; `comfy-aimdo 0.5.3`, `comfy-kitchen 0.2.33`, frontend 1.51.10 |
| Last known good | v0.34.6 (`comfy-aimdo 0.4.15`) — 116 prompts / 12 h on 2026-09-08, 554 dynamic loads + 63 partial loads, zero issues |
| GPU | NVIDIA GeForce RTX 3050 **6 GB**, driver 616.92 (`nvlddmkm.sys` built 2026-09-04), WDDM 3.2, HAGS on |
| Also present | AMD Radeon iGPU (Ryzen 5 9600X) driving the monitors; RTX 3050 is headless (compute only). Confirmed from the kernel dump that the faulting adapter is the **NVIDIA** one (see below). |
| OS | Windows 11 Pro 26200.9445 (KB5124008 installed 2026-09-09 — same day as the ComfyUI update, so I can't fully exclude an OS-side regression; noted for honesty) |
| RAM | 64 GB DDR5 |
| Python / torch | 3.13.12, torch 2.12.1+cu130 (Desktop standalone env `v0.29.0-env1`), `cudaMallocAsync` enabled, `NORMAL_VRAM` |
| Launch args | `--enable-manager` (default Desktop launch) |

## Crashes

| # | Time (local) | Bugcheck | Faulting function |
|---|---|---|---|
| 1 | 2026-09-10 12:12 | `0x50 PAGE_FAULT_IN_NONPAGED_AREA` | `dxgmms2!GetVidMmGlobalAllocFromOwner+0xf` |
| 2 | 2026-09-10 14:46 | `0x7E SYSTEM_THREAD_EXCEPTION_NOT_HANDLED (c0000005)` | `dxgmms2!GetVidMmGlobalAllocFromOwner+0x12` |
| 3 | 2026-09-10 ~16:34 | `0x7E (c0000005)` | `dxgmms2!VIDMM_PAGE_DIRECTORY::ExpandLargePagePteWithFix+0x179` |
| 4 | 2026-09-10 ~19:18 | `0x50` | `dxgmms2!GetVidMmGlobalAllocFromOwner+0xf` (same instruction as #1) |

Identical stack in all four (from `kd !analyze -v`, Microsoft public symbols):

```
dxgmms2!GetVidMmGlobalAllocFromOwner <- reads freed "owner" pointer
dxgmms2!VIDMM_PAGE_DIRECTORY::ExpandLargePagePteWithFix
dxgmms2!VIDMM_PAGE_DIRECTORY::ExpandLargePagePte
dxgmms2!VIDMM_PAGE_DIRECTORY::CommitVirtualAddressRange
dxgmms2!CVirtualAddressAllocator::CommitVirtualAddressRange
dxgmms2!CommitAllocationVirtualAddressCb
dxgmms2!VIDMM_PHYSICAL_ADAPTER::IteratePhysicalMemoryRanges
dxgmms2!CommitVirtualAddressRangesInList
dxgmms2!VIDMM_GLOBAL::PageInOneAllocation <- VidMm paging an allocation into GPU-visible memory
dxgmms2!VIDMM_GLOBAL::ProcessDeferredCommand
dxgmms2!VidMmWorkerThreadProc
nt!PspSystemThreadStartup
```

Kernel dump (crash #4, 3.9 GB automatic dump) details:

- `GetVidMmGlobalAllocFromOwner` executes `mov rax,[rdx]` with `rdx = ffffac839aa5e1c0`; `!pte` on that address: **PDE = 0, "not valid"** — the allocation's owner object had already been freed. i.e. a **use-after-free race between an allocation being destroyed and VidMm's deferred page-in of it**.
- The `DXGADAPTER` reachable from the VidMm objects of the failing operation has a DDI table with **126 function pointers into `nvlddmkm.sys` and 0 into `amdkmdag.sys`** → the NVIDIA adapter (the one ComfyUI uses), not the iGPU.
- Context process: `System` (VidMm worker thread); 7 `python.exe` (ComfyUI server) alive at crash time.

## ComfyUI log at the moment of the crashes

Crash #3 (v0.35.0) and crash #4 (v0.35.1) both end on the **same line** of the same workflow (Z-Image → Lumina2, run after an LTXAV video generation):

```
[2026-09-10 19:17:49.525] got prompt
[2026-09-10 19:17:49.907] Using RAM pressure cache.
[2026-09-10 19:17:50.959] Requested to load ZImageTEModel_
[2026-09-10 19:17:51.070] Model ZImageTEModel_ prepared for dynamic VRAM loading. 7671MB Staged. 0 patches attached. Force pre-loaded 145 weights: 383 KB.
[2026-09-10 19:18:08.751] 0 models unloaded.
[2026-09-10 19:18:08.777] Model ZImageTEModel_ prepared for dynamic VRAM loading. 7671MB Staged. 0 patches attached. Force pre-loaded 145 weights: 383 KB.
[2026-09-10 19:18:09.810] model_type FLOW
[2026-09-10 19:18:09.818] Requested to load Lumina2
[2026-09-10 19:18:10.127] Model Lumina2 prepared for dynamic VRAM loading. 11738MB Staged. 0 patches attached. Force pre-loaded 205 weights: 1045 KB.
<-- BSOD here (crash #4). Crash #3 log ends on the identical line at 16:33:52.
```

Earlier in the same session, LTXAV had been staged at `20484MB Staged` (two 440 s video generations completed fine); the crash comes when the next workflow stages ~19 GB of new weights while the previous ones are being evicted — i.e. the point of maximum allocate/free churn on a 6 GB card.

Note the v0.35 log shows **0 `loaded partially`** lines in these sessions, whereas the v0.34.6 log for the same kind of work shows 63 — the loading strategy changed with the `comfy-aimdo` 0.5 bump.

## What I've ruled out

- Not power / thermal / storage / RAM at random: zero WHEA events in 14 days, all SSDs healthy, identical instruction in 4 dumps (random memory corruption doesn't do that).
- Removed a third-party virtual display driver (IddSampleDriver-based) → crash #4 happened without it.
- Reinstalling the NVIDIA driver (same version) → crash #3 happened 77 min later.

## Workaround / status

- Rolled back to **v0.34.6** via ComfyUI Desktop snapshot and disabled auto-update; also running with `--disable-dynamic-vram` for now.
- Startup on v0.35.1 with `--disable-dynamic-vram` prints: *"Dynamic vram disabled with argument. If you have any issues with dynamic vram enabled please give us a detailed reports as this argument will be removed soon"* — please **keep that escape hatch** at least until the 0.5.x loader is safe on small-VRAM WDDM GPUs.

## Suggested things to look at

- Whatever changed in `comfy-aimdo` 0.4.15 → 0.5.3 around freeing/re-staging weight buffers while a page-in of the same allocation is still queued (VidMm deferred commands). The kernel side clearly sees an allocation owner destroyed while its page-in is pending; on Windows that's a classic way to hit `dxgmms2` bugs (Microsoft had a very similar `dxgmms2` race regression in Jan 2026, fixed in KB5077181).
- Interaction with `cudaMallocAsync` + pinned memory under heavy eviction on 6 GB cards.

I can provide the four minidumps, the full kernel dump `!analyze -v` output, `lm` module list, and the complete ComfyUI logs of the crashing sessions on request. Happy to test a patched `comfy-aimdo` build.

Contributor guide

Open the contributing guide

Research direction

Start by comparing the dynamic VRAM loader behavior between comfy-aimdo 0.4.15 and 0.5.3, focusing on allocation freeing and re-staging during model eviction. Reproduce the Z-Image → Lumina2 workflow on a 6 GB NVIDIA GPU, with and without --disable-dynamic-vram, and use the supplied kernel dumps and ComfyUI logs to determine whether the crash trigger is eliminated.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, operating-systems, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.