huggingface / huggingface/diffusers

Can diffusers support loading and running FLUX with fp8 ?

Open
#11,580 6 comments 0 reactions 0 assignees View on GitHub
stale
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

This is how I use diffusers to load flux model:
```
import torch
from diffusers import FluxPipeline
pipe = FluxPipeline.from_pretrained(
"/ckptstorage/repo/pretrained_weights/black-forest-labs/FLUX.1-dev",
torch_dtype=torch.float16,
)
device = torch.device(f"cuda:{device_number}" if torch.cuda.is_available() else "cpu")
pipe = pipe.to(device)
```
it consumes about 75 seconds on my computer with A800 GPU.
But I found in comfyui, it only need 22 seconds to load flux model, but it load the fp8 model.
Can diffusers load flux fp8 model ?
or is there any other speed up method ?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.