huggingface / huggingface/diffusers

`from_pipe` converts pipelines to float32 by default

Open
#12,754 13 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

### Describe the bug

Pipelines passed to `from_pipe()` are converted to float32 unless `torch_dtype` is specified, leading to higher memory usage and slower inference.

### Reproduction
```python
import torch
from diffusers import StableDiffusionPipeline, StableDiffusionImg2ImgPipeline

pipe = StableDiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16).to("cuda")
print(f"Before: {pipe.dtype} - {torch.cuda.memory_allocated() // 1048576} MB")

i2i = StableDiffusionImg2ImgPipeline.from_pipe(pipe)
print(f"After: {pipe.dtype} - {torch.cuda.memory_allocated() // 1048576} MB")
```
### Logs
```
Loading pipeline components...: 0%| | 0/7 [00:00

Contributor guide

Open the contributing guide

Research direction

Start at the `from_pipe()` entry point and reproduce the conversion with the StableDiffusionPipeline and StableDiffusionImg2ImgPipeline example from the issue. Compare the pipeline dtype and CUDA memory before and after conversion. Done means a pipeline passed through `from_pipe()` retains the source dtype by default without requiring `torch_dtype`.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.