huggingface / huggingface/diffusers
enable_attention_slicing give NaN results for SDXL on MPS
- Langage dominant
- Python
- Étoiles
- 34.5k
- Forks
- 7.3k
- Merge moyen
- 3 j 3 h
- PR mergées (30 j)
- 91
Description
### Describe the bug
If I call pipe.enable_attention_slicing I get NaN's returned when output type is 'latent' and a value error for image output.
The error is....
```py
/Volumes/SSD2TB/AI/Diffusers/lib/python3.11/site-packages/diffusers/image_processor.py:147: RuntimeWarning: invalid value encountered in cast
images = (images * 255).round().astype("uint8")
```
printing the latent gives
```py
$ python tas.py
Loading pipeline components...: 100%|█████████████████████████████████████████████████████| 6/6 [00:00<00:00, 8.89it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:10<00:00, 5.26s/it]
tensor([[[[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
...,
...
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan]]]], device='mps:0',
dtype=torch.float16)
```
commenting out the call to enable_attention_slicing gives non NaN latest and a proper image.
I've tested this on the current release version and the current HEAD version
### Reproduction
```py
from diffusers import DiffusionPipeline
import torch
model_id = "stabilityai/stable-diffusion-xl-base-1.0"
pipe = DiffusionPipeline.from_pretrained(model_id, variant="fp16",
torch_dtype=torch.float16)
pipe.to(device="mps", torch_dtype=torch.float16)
pipe.enable_attention_slicing()
prompt = "analog film photo Butterflies in a jungle, cold color palette, vivid colors, detailed, 8k, 35mm photo, Kodachrome, Lomography, highly detailed"
negative_prompt = "painting, drawing, illustration, glitch, deformed, mutated, cross-eyed, ugly, disfigured"
images = pipe(
prompt=prompt,
negative_prompt=negative_prompt,
num_inference_steps=2,
guidance_scale=7,
output_type='latent'
).images
print(images)
```
comment out or delete `pipe.enable_attention_slicing()` and the script works as expected.
### Logs
```shell
(Diffusers) $ python tas.py
Loading pipeline components...: 100%|█████████████████████████████████████████████████████| 7/7 [00:00<00:00, 10.57it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:10<00:00, 5.34s/it]
tensor([[[[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
...,
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan]],
[[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
...,
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan]],
[[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
...,
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan]],
[[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
...,
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan]]]], device='mps:0',
dtype=torch.float16)
(Diffusers) $
```
### System Info
- 🤗 Diffusers version: 0.33.0.dev0
- Platform: macOS-15.3.2-arm64-arm-64bit
- Running on Google Colab?: No
- Python version: 3.11.10
- PyTorch version (GPU?): 2.6.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Huggingface_hub version: 0.27.1
- Transformers version: 4.50.3
- Accelerate version: 0.34.2
- PEFT version: not installed
- Bitsandbytes version: not installed
- Safetensors version: 0.4.5
- xFormers version: not installed
- Accelerator: Apple M3
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No
### Who can help?
@pcuenca
I'm assuming its more MPS related than SDXL.
Guide de contribution
Ouvrir le guide de contribution
Piste de recherche
Commencez par la reproduction SDXL fournie sur le périphérique MPS et suivez pipe.enable_attention_slicing dans l’implémentation de l’attention de Diffusers. Utilisez image_processor.py:147 comme symptôme signalé, puis vérifiez que le même script produit des valeurs latentes finies et une image valide avec l’attention slicing activé, sans l’erreur signalée.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- macos, python, pytorch
- Domaine
- machine-learning
- Type d'issue
- Bug
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Activité
- Active
- Clarté
- Plutôt claire
- Accessibilité débutants
- 52/100