MiniMax Music 3 crashes with CUDA Graph + Dynamic VRAM
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
### Custom Node Testing
- [ ] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)
### Expected Behavior
MiniMax Music 3 should generate audio successfully when using a quantized GGUF text encoder with `CLIPLoaderGGUFDynamicVRAM`.
CUDA Graph capture should either work safely with the dynamically offloaded GGUF weights, or be disabled automatically for this incompatible configuration.
### Actual Behavior
The model loads successfully and AR sampling begins, but generation crashes during Q4_0 dequantization with:
```text
RuntimeError: Cannot copy between CPU and CUDA tensors during CUDA graph capture unless the CPU tensor is pinned.
```
The traceback ends in:
```text
ComfyUI-GGUF/dequant.py -> dequantize_blocks_Q4_0()
```
### Steps to Reproduce
1. Use a Windows portable NVIDIA ComfyUI build.
2. Use an RTX 4060 with 8 GB VRAM.
3. Load MiniMax Music 3 with:
- `minimax_music3_text_encoder_pruned_Q4_0.gguf`
- `CLIPLoaderGGUFDynamicVRAM`
4. Run a MiniMax Music 3 workflow and start generation.
5. Wait until AR sampling reaches the autoregressive decode stage.
The model loads correctly and sampling starts, then fails during Q4_0 dequantization while Dynamic VRAM and CUDA Graph capture are active.
A related discussion and GGUF-side investigation are available in:
[https://github.com/molbal/ComfyUI-GGUF/issues/17](https://github.com/molbal/ComfyUI-GGUF/issues/17)
### Debug Logs
```powershell
@6TbhZ please add the logs here
```
### Other
Environment:
- ComfyUI: 0.34.0
- Platform: Windows portable NVIDIA build
- GPU: NVIDIA RTX 4060, 8 GB VRAM
- PyTorch: 2.13.0
- CUDA: 13.0
- Text encoder: `minimax_music3_text_encoder_pruned_Q4_0.gguf`
- Loader: `CLIPLoaderGGUFDynamicVRAM`
The issue appears to be caused by CUDA Graph capture being enabled while GGUF weights are dynamically transferred/offloaded and dequantized. The relevant ComfyUI paths are:
- `comfy/ldm/minimax_music/ar.py`
- `self.model.graph_dynamic_vbar_blocks = True`
- depth decoder prefetch with `enable_graph=True`
- `comfy/model_prefetch.py`
- `comfy/text_encoders/llama.py`
As a workaround, setting both MiniMax CUDA Graph paths to false makes the same workflow run reliably:
```python
self.model.graph_dynamic_vbar_blocks = False
```
and:
```python
enable_graph=False
```
---
Why this is a ComfyUI bug, not a GGUF-loader bug
The GGUF loader is not crashing merely because it dequantizes Q4_0 data. The same Q4_0 model and loader work correctly when ComfyUI executes the operation normally.
The crash occurs only when **ComfyUI enables CUDA Graph capture** around MiniMax Music 3 execution:
```text
Cannot copy between CPU and CUDA tensors during CUDA graph capture
unless the CPU tensor is pinned
```
This is a PyTorch CUDA Graph constraint. During graph capture, ComfyUI causes the GGUF loader’s dynamically offloaded CPU tensor to be copied/dequantized into CUDA memory. That CPU tensor is not pinned, so PyTorch correctly rejects the operation.
The decisive evidence is the workaround: without changing the GGUF file, loader, dequantizer, or model, generation succeeds when these **ComfyUI-owned graph controls** are disabled:
```python
self.model.graph_dynamic_vbar_blocks = False
```
and:
```python
enable_graph=False
```
Therefore:
1. The GGUF file is valid and loads successfully.
2. Q4_0 dequantization works outside CUDA Graph capture.
3. Dynamic VRAM operation works when ComfyUI does not capture these operations.
4. Disabling ComfyUI’s two CUDA Graph paths fixes generation.
5. The failure is caused by ComfyUI selecting an execution mode whose memory requirements are incompatible with dynamically offloaded GGUF tensors.
The traceback identifies the operation where PyTorch detects the invalid CPU-to-CUDA copy; it does **not** prove that the loader caused the bug. The loader is supplying a valid CPU-backed quantized tensor. ComfyUI is responsible for deciding whether that tensor movement and dequantization occur inside CUDA Graph capture.
A custom node may be required to reproduce the issue because it provides GGUF quantized weights, but that does not make the custom node the owner of the failure. The problematic behavior is ComfyUI’s unconditional MiniMax CUDA Graph/prefetch path not accounting for externally managed, CPU-backed quantized weights.
The appropriate fix is for ComfyUI to detect this incompatible weight configuration and disable the affected graph paths, or otherwise ensure that all captured inputs use stable CUDA addresses and pinned host memory.
Contributor guide
Research direction
Start with comfy/ldm/minimax_music/ar.py, then trace the CUDA Graph and prefetch behavior through comfy/model_prefetch.py and comfy/text_encoders/llama.py. Reproduce the MiniMax Music 3 workflow with CLIPLoaderGGUFDynamicVRAM and compare it with both graph paths disabled. Done means the incompatible configuration no longer crashes during Q4_0 dequantization while compatible graph execution remains functional.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- backend, machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100