Comfy-Org / Comfy-Org/ComfyUI

Complete system freeze (no dump, requires hard power cycle) when running FireRed → MiniMax H3 sequence with dual NVIDIA GPU system

Open
#15,360 2 comments 2 reactions 0 assignees View on GitHub
duplicate
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

Honest preamble: I'm writing this post because Claude insists it might be helpful, and indeed it wrote this, after a whole day of me and it trying to figure out the problem with no success. Honestly, I don't think it's helpful. I did, however, carefully read the entire post and corrected where Claude had made a mistake.

## Bug Description

Running a single ComfyUI workflow that includes both **FireRed** image editing and **MiniMax H3** video generation causes an **unrecoverable kernel-level freeze** on Windows 11 with a dual NVIDIA GPU system. The system must be hard-powered off; no recovery occurs.

This is **not** an OOM error, a Python crash, or a ComfyUI-level failure. The Windows kernel itself freezes: no application responds, hard power cycle required.

The freeze is **not immediately reproducible** — it occurs intermittently, typically after 2–5 successful runs of the same workflow in the same ComfyUI session.

## Environment

| | |
|---|---|
| **ComfyUI** | 0.30.0 |
| **Frontend** | v1.47.12 |
| **Templates** | v0.11.27 |
| **ComfyUI-Manager** | V4.2.2 |
| **PyTorch** | 2.13.0+cu130 |
| **CUDA** | 13.0 |
| **Python** | embedded |
| **OS** | Windows 11 build 26200 |
| **GPU 0 (active)** | ASUS TUF GeForce RTX 4090 Gaming OC — 24 GB VRAM |
| **GPU 1 (present, idle)** | ASUS ROG STRIX GeForce RTX 3090 Gaming OC — 24 GB VRAM |
| **CPU** | AMD Ryzen Threadripper 3990X (64c/128t) |
| **Motherboard** | ASUS ROG ZENITH II EXTREME ALPHA (TRX40) |
| **NVIDIA Driver** | Studio Driver 32.0.16.1062 |
| **CUDA device** | `set_cuda_devices` → GPU 0 (4090) only |
| **Power limit** | 70% (`-setPowerTarget:0,70`) |
| **Core clock offset** | -500 MHz |

## Steps to Reproduce

1. Build a single ComfyUI workflow containing both **FireRed** (image-to-image) and **MiniMax H3** (image-to-video) nodes, running sequentially on GPU 0 (RTX 4090)
2. Run the workflow successfully 2–5 times within the same ComfyUI session
3. After an unpredictable number of successful runs, the system freezes during workflow execution

**Reproducibility:** intermittent — typically occurs within the first 2–5 runs of a session. Not reproducible on the first run; accumulates over repeated executions without restarting ComfyUI.

## Observed Behavior

Two distinct freeze modes have been observed:

**Mode 1 — Full kernel freeze (harder variant):**
- Complete system freeze: no mouse movement, no display update
- GPU fans remain at full speed
- No BSOD, no minidump written to `C:\WINDOWS\Minidump\`
- Hard power cycle required

**Mode 2 — Partial kernel freeze (softer variant, more common):**
- Mouse cursor still moves and changes shape correctly (e.g. resize cursor appears at window edges)
- However: no window responds to clicks, no Z-order changes, no UI interaction of any kind works
- System appears superficially alive but is completely unresponsive at the application/kernel scheduler level
- Also requires hard power cycle to recover

When the freeze occurs slightly earlier in the process, a `DPC_WATCHDOG_VIOLATION (0x133)` bugcheck is generated with parameters:
- `P1=0x1` (cumulative timeout)
- `P2=0x1e00` (7680ms stall)
- `P3=0xfffff800a43c43c8` (address within `ntoskrnl.exe`)

## What This Is NOT

- Not an OOM: PyTorch/ComfyUI-level OOM would raise a Python exception and fail gracefully
- Not a ComfyUI crash: the OS itself is unresponsive
- Not a hardware overload: GPU 0 is running at 70% power limit with -500 MHz core clock offset; GPU 1 is physically present but completely idle (not enumerated for compute)
- Not reproducible with: Blender, Houdini, Unreal Engine, NVIDIA Omniverse, FFmpeg — all of which use the GPU heavily without issue

## Hypothesis

MiniMax H3 ships with **custom CUDA kernels** and **int8 convrot quantization**. After multiple sequential executions within the same session, GPU memory state accumulates in a way that triggers a deadlock inside `nvlddmkm.sys` (NVIDIA Windows kernel-mode driver). The intermittent nature suggests a race condition or resource exhaustion that builds up over repeated runs rather than a deterministic bug on first execution.

The presence of a second GPU (RTX 3090) on the PCIe bus — even when idle and not assigned to compute — may be a contributing factor, as the NVIDIA driver enumerates both devices regardless of `set_cuda_devices`.

`torch.cuda.empty_cache()` and `torch.cuda.synchronize()` between workflow nodes do not prevent the freeze.

## Workaround Attempted

- `torch.cuda.empty_cache()` between model executions → no effect
- `torch.cuda.synchronize()` between model executions → no effect
- Reducing GPU power to 70% and core clock to -500 MHz → no effect

**Only known workaround:** restart ComfyUI completely between sessions (not acceptable for production workflows).

## Additional Notes

- MiniMax H3 was released 2026-08-02. This may be one of the first reports of this specific combined workflow on a dual-GPU Windows workstation.
- A minidump from an occurrence of the related `DPC_WATCHDOG_VIOLATION (0x133)` crash is available and can be attached if useful.
- The issue appears to be in the CUDA/driver path rather than ComfyUI Python code. May warrant forwarding to NVIDIA.

Contributor guide

Open the contributing guide

Research direction

No ComfyUI file, test, or entry point is named. Begin by reproducing the FireRed→MiniMax H3 workflow on the stated Windows 11 dual-GPU setup, then inspect the available 0x133 dump and compare runs with and without restarting the session. Done would require isolating a ComfyUI-triggered cause or documenting evidence that the failure belongs to the CUDA/NVIDIA driver path for escalation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, operating-systems, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.