Comfy-Org / Comfy-Org/ComfyUI

MiniMax Music 3 is unusable on MPS: AR conditioning stage runs on CPU (5.7 s/it, ~2.4 h per song)

Open Beginner friendly
#15,640 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Summary

On Apple Silicon, `text_encoder_device()` always returns `cpu`, because MPS reports `VRAMState.SHARED`, which falls through to the CPU branch:

```python
def text_encoder_device():
if args.gpu_only:
return get_torch_device()
elif vram_state in (VRAMState.HIGH_VRAM, VRAMState.NORMAL_VRAM) or comfy.memory_management.aimdo_enabled:
if should_use_fp16(prioritize_performance=False):
return get_torch_device()
...
else:
return torch.device("cpu") # <- SHARED always lands here
```

For a one-shot CLIP/T5 encode this was never noticeable. MiniMax Music 3 changed the cost profile: its "text encoder" stage is an autoregressive transformer that samples ~12.5 tokens per second of audio (1501 steps for a 120 s song). Running that on CPU makes the model effectively unusable on Macs.

### Measurements

M-series MacBook Pro, 68 GB unified memory, torch 2.14.0.dev (MPS), fp16 DiT + pruned int8_convrot text encoder from `Comfy-Org/MiniMax-Music-3`, 120 s target duration:

| device | AR sampling | full AR stage (1501 steps) |
|---|---|---|
| cpu (current behavior) | 5.69 s/it | ~2 h 22 min |
| mps | 1.19 it/s (0.84 s/it) | ~21 min |

6.8x. Identical workflow, only the text encoder load device differs.

### Suggested fix

Adding `VRAMState.SHARED` to the GPU branch is a one-word change and is what I'm running locally:

```python
elif vram_state in (VRAMState.HIGH_VRAM, VRAMState.NORMAL_VRAM, VRAMState.SHARED) or comfy.memory_management.aimdo_enabled:
```

On unified memory the GPU placement costs no additional RAM versus CPU, `text_encoder_offload_device()` still returns `cpu` so eviction behavior is unchanged, and the existing `should_use_fp16(prioritize_performance=False)` guard keeps weak GPUs on the CPU path. `--gpu-only` is a workaround but is much blunter — it also pins the offload device, disabling model eviction for the whole session, which hurts workflows that are near the memory ceiling.

If blanket-changing SHARED placement for all text encoders is considered too risky, alternatives that would also solve it:

- keep `default` = cpu on SHARED, but add a `gpu` option to `CLIPLoader`'s `device` widget (it currently only offers `default`/`cpu`, so there is no per-workflow escape hatch today)
- gate the GPU placement on model size or on the CLIP type (`minimax`), so only heavy AR conditioning models move

Happy to send whichever variant as a PR.

Contributor guide

Open the contributing guide

Research direction

Locate text_encoder_device() and the VRAMState handling shown in the issue. Check the existing should_use_fp16 guard and text_encoder_offload_device() behavior before making the narrowly scoped placement change. Done means MiniMax Music 3's AR stage uses MPS on Apple Silicon without changing offload behavior, while weak-GPU fallback remains intact; verify with relevant tests or a measured workflow.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
ai, backend, performance
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.