huggingface / huggingface/diffusers

MiniMax-H3: float64 rotary position grid cannot be moved to MPS (`Cannot convert a MPS Tensor to float64`)

Offen Anfängerfreundlich
#14,639 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug
Vorherrschende Sprache
Python
Sterne
34.5k
Forks
7.3k
Ø Merge
3 T. 3 Std.
Gemergte PRs (30 T.)
91

Beschreibung

### Describe the bug

Both MiniMax-H3 layout blocks build their rotary position grid in float64 on CPU (deliberately — the grid has to reproduce the released model's fp64 coordinates exactly) and then move it to the execution device with a plain `.to(device)`:

- `MiniMaxH3PrepareLayoutStep`, `before_denoise.py` line 444 (v0.40.0): `block_state.position_ids = position_ids.to(device)`
- `MiniMaxH3Ref2VAPrepareLayoutStep`, line 768: same statement

Metal has no float64, so on MPS every workflow (`t2va`, `fl2va`, `ref2va`) dies at that line before the first denoising step. No MPS-specific handling exists anywhere on that path.

### Reproduction

### Reproduction

The failing operation in isolation (this is exactly what the layout step does):

```python
import torch
position_ids = torch.zeros(8, 3, dtype=torch.float64) # what MiniMaxH3PrepareLayoutStep builds
position_ids.to("mps")
```

With the real pipeline: `MiniMaxH3ModularPipeline` on `device="mps"`, any workflow, fails inside `prepare_layout` with the same error.

### Logs

```
TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.
```

### Proposed fix

Keep the fp64 computation untouched and drop the dtype only at the device transfer, only for MPS (two sites):

```python
# before_denoise.py, MiniMaxH3PrepareLayoutStep and MiniMaxH3Ref2VAPrepareLayoutStep
if device is not None and torch.device(device).type == "mps":
block_state.position_ids = position_ids.to(device, dtype=torch.float32)
else:
block_state.position_ids = position_ids.to(device)
```

Precision impact, measured on the grids the code actually produces (`_spatial_position_grid`, `_temporal_position_grid`, audio rows) for a large layout — 64×64 latent frame with patch 2 (1024×1024 output), 102 latent frames (~15 s), 4000 audio latents:

| grid | max \|coord\| | max abs error after fp32 cast |
|---|---:|---:|
| spatial h / w | 31.0 | 0 (exact) |
| temporal | 568.3 | 2.0e-5 |
| audio time | 4567.3 | 1.6e-4 |

Worst case is 1.6e-4 rad of rotary phase at the highest frequency (`inv_freq = 1`), about 24× below the bf16 precision (3.9e-3) the transformer itself runs in. With this patch applied locally, all three workflows run end to end on an M5 Max (t2va, fl2va with keyframes, ref2va with up to 12 references), outputs reproducible across runs. Happy to open a PR if this shape of fix is acceptable.

### Logs

```shell

```

### System Info

- 🤗 Diffusers version: 0.40.0 (sites verified against the v0.40.0 tag; also present on main)
- Platform: macOS-26.6.2-arm64-arm-64bit-Mach-O
- Python version: 3.14.5
- PyTorch version (GPU?): 2.14.0.dev20260808 (MPS)
- Transformers version: 5.14.1
- Accelerate version: 1.14.0
- Accelerator: Apple M5 Max, 128 GB unified memory
- Using GPU in script?: yes (MPS)
- Using distributed or parallel set-up in script?: no

### Who can help?

@apolinario (MiniMax-H3 integration, #14355)

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Start in before_denoise.py at MiniMaxH3PrepareLayoutStep around line 444 and MiniMaxH3Ref2VAPrepareLayoutStep around line 768, keeping the fp64 grid construction unchanged. Check the device-transfer behavior for MPS and validate that the t2va, fl2va, and ref2va workflows complete without the float64 conversion error.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
macos, python, pytorch
Bereich
machine-learning
Issue-Typ
Bug
Schwierigkeit
2/5
Geschätzter Aufwand
1-3 Stunden
Aktivitätsstatus
Aktiv
Klarheit
Klar beschrieben
Anfängerfreundlichkeit
88/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.