MiniMax Music 3 is unusable on MPS: AR conditioning stage runs on CPU (5.7 s/it, ~2.4 h per song)
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
### Summary
On Apple Silicon, `text_encoder_device()` always returns `cpu`, because MPS reports `VRAMState.SHARED`, which falls through to the CPU branch:
```python
def text_encoder_device():
if args.gpu_only:
return get_torch_device()
elif vram_state in (VRAMState.HIGH_VRAM, VRAMState.NORMAL_VRAM) or comfy.memory_management.aimdo_enabled:
if should_use_fp16(prioritize_performance=False):
return get_torch_device()
...
else:
return torch.device("cpu") # <- SHARED always lands here
```
For a one-shot CLIP/T5 encode this was never noticeable. MiniMax Music 3 changed the cost profile: its "text encoder" stage is an autoregressive transformer that samples ~12.5 tokens per second of audio (1501 steps for a 120 s song). Running that on CPU makes the model effectively unusable on Macs.
### Measurements
M-series MacBook Pro, 68 GB unified memory, torch 2.14.0.dev (MPS), fp16 DiT + pruned int8_convrot text encoder from `Comfy-Org/MiniMax-Music-3`, 120 s target duration:
| device | AR sampling | full AR stage (1501 steps) |
|---|---|---|
| cpu (current behavior) | 5.69 s/it | ~2 h 22 min |
| mps | 1.19 it/s (0.84 s/it) | ~21 min |
6.8x. Identical workflow, only the text encoder load device differs.
### Suggested fix
Adding `VRAMState.SHARED` to the GPU branch is a one-word change and is what I'm running locally:
```python
elif vram_state in (VRAMState.HIGH_VRAM, VRAMState.NORMAL_VRAM, VRAMState.SHARED) or comfy.memory_management.aimdo_enabled:
```
On unified memory the GPU placement costs no additional RAM versus CPU, `text_encoder_offload_device()` still returns `cpu` so eviction behavior is unchanged, and the existing `should_use_fp16(prioritize_performance=False)` guard keeps weak GPUs on the CPU path. `--gpu-only` is a workaround but is much blunter — it also pins the offload device, disabling model eviction for the whole session, which hurts workflows that are near the memory ceiling.
If blanket-changing SHARED placement for all text encoders is considered too risky, alternatives that would also solve it:
- keep `default` = cpu on SHARED, but add a `gpu` option to `CLIPLoader`'s `device` widget (it currently only offers `default`/`cpu`, so there is no per-workflow escape hatch today)
- gate the GPU placement on model size or on the CLIP type (`minimax`), so only heavy AR conditioning models move
Happy to send whichever variant as a PR.
Contributor guide
Research direction
Locate text_encoder_device() and the VRAMState handling shown in the issue. Check the existing should_use_fp16 guard and text_encoder_offload_device() behavior before making the narrowly scoped placement change. Done means MiniMax Music 3's AR stage uses MPS on Apple Silicon without changing offload behavior, while weak-GPU fallback remains intact; verify with relevant tests or a measured workflow.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- ai, backend, performance
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100