Comfy-Org / Comfy-Org/ComfyUI

MiniMax Music 3 crashes with CUDA Graph + Dynamic VRAM

Open
#16,002 3 comments 0 reactions 0 assignees View on GitHub
Potential Bug
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Custom Node Testing

- [ ] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Expected Behavior

MiniMax Music 3 should generate audio successfully when using a quantized GGUF text encoder with `CLIPLoaderGGUFDynamicVRAM`.

CUDA Graph capture should either work safely with the dynamically offloaded GGUF weights, or be disabled automatically for this incompatible configuration.

### Actual Behavior

The model loads successfully and AR sampling begins, but generation crashes during Q4_0 dequantization with:

```text
RuntimeError: Cannot copy between CPU and CUDA tensors during CUDA graph capture unless the CPU tensor is pinned.

```

The traceback ends in:

```text
ComfyUI-GGUF/dequant.py -> dequantize_blocks_Q4_0()

```

### Steps to Reproduce

1. Use a Windows portable NVIDIA ComfyUI build.
2. Use an RTX 4060 with 8 GB VRAM.
3. Load MiniMax Music 3 with:
- `minimax_music3_text_encoder_pruned_Q4_0.gguf`
- `CLIPLoaderGGUFDynamicVRAM`
4. Run a MiniMax Music 3 workflow and start generation.
5. Wait until AR sampling reaches the autoregressive decode stage.

The model loads correctly and sampling starts, then fails during Q4_0 dequantization while Dynamic VRAM and CUDA Graph capture are active.

A related discussion and GGUF-side investigation are available in:
[https://github.com/molbal/ComfyUI-GGUF/issues/17](https://github.com/molbal/ComfyUI-GGUF/issues/17)

### Debug Logs

```powershell
@6TbhZ please add the logs here
```

### Other

Environment:

- ComfyUI: 0.34.0
- Platform: Windows portable NVIDIA build
- GPU: NVIDIA RTX 4060, 8 GB VRAM
- PyTorch: 2.13.0
- CUDA: 13.0
- Text encoder: `minimax_music3_text_encoder_pruned_Q4_0.gguf`
- Loader: `CLIPLoaderGGUFDynamicVRAM`

The issue appears to be caused by CUDA Graph capture being enabled while GGUF weights are dynamically transferred/offloaded and dequantized. The relevant ComfyUI paths are:

- `comfy/ldm/minimax_music/ar.py`
- `self.model.graph_dynamic_vbar_blocks = True`
- depth decoder prefetch with `enable_graph=True`
- `comfy/model_prefetch.py`
- `comfy/text_encoders/llama.py`

As a workaround, setting both MiniMax CUDA Graph paths to false makes the same workflow run reliably:

```python
self.model.graph_dynamic_vbar_blocks = False

```

and:

```python
enable_graph=False

```

---

Why this is a ComfyUI bug, not a GGUF-loader bug

The GGUF loader is not crashing merely because it dequantizes Q4_0 data. The same Q4_0 model and loader work correctly when ComfyUI executes the operation normally.

The crash occurs only when **ComfyUI enables CUDA Graph capture** around MiniMax Music 3 execution:

```text
Cannot copy between CPU and CUDA tensors during CUDA graph capture
unless the CPU tensor is pinned

```

This is a PyTorch CUDA Graph constraint. During graph capture, ComfyUI causes the GGUF loader’s dynamically offloaded CPU tensor to be copied/dequantized into CUDA memory. That CPU tensor is not pinned, so PyTorch correctly rejects the operation.

The decisive evidence is the workaround: without changing the GGUF file, loader, dequantizer, or model, generation succeeds when these **ComfyUI-owned graph controls** are disabled:

```python
self.model.graph_dynamic_vbar_blocks = False

```

and:

```python
enable_graph=False

```

Therefore:

1. The GGUF file is valid and loads successfully.
2. Q4_0 dequantization works outside CUDA Graph capture.
3. Dynamic VRAM operation works when ComfyUI does not capture these operations.
4. Disabling ComfyUI’s two CUDA Graph paths fixes generation.
5. The failure is caused by ComfyUI selecting an execution mode whose memory requirements are incompatible with dynamically offloaded GGUF tensors.

The traceback identifies the operation where PyTorch detects the invalid CPU-to-CUDA copy; it does **not** prove that the loader caused the bug. The loader is supplying a valid CPU-backed quantized tensor. ComfyUI is responsible for deciding whether that tensor movement and dequantization occur inside CUDA Graph capture.

A custom node may be required to reproduce the issue because it provides GGUF quantized weights, but that does not make the custom node the owner of the failure. The problematic behavior is ComfyUI’s unconditional MiniMax CUDA Graph/prefetch path not accounting for externally managed, CPU-backed quantized weights.

The appropriate fix is for ComfyUI to detect this incompatible weight configuration and disable the affected graph paths, or otherwise ensure that all captured inputs use stable CUDA addresses and pinned host memory.

Contributor guide

Open the contributing guide

Research direction

Start with comfy/ldm/minimax_music/ar.py, then trace the CUDA Graph and prefetch behavior through comfy/model_prefetch.py and comfy/text_encoders/llama.py. Reproduce the MiniMax Music 3 workflow with CLIPLoaderGGUFDynamicVRAM and compare it with both graph paths disabled. Done means the incompatible configuration no longer crashes during Q4_0 dequantization while compatible graph execution remains functional.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
backend, machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.