huggingface / huggingface/diffusers
Can diffusers support loading and running FLUX with fp8 ?
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
This is how I use diffusers to load flux model:
```
import torch
from diffusers import FluxPipeline
pipe = FluxPipeline.from_pretrained(
"/ckptstorage/repo/pretrained_weights/black-forest-labs/FLUX.1-dev",
torch_dtype=torch.float16,
)
device = torch.device(f"cuda:{device_number}" if torch.cuda.is_available() else "cpu")
pipe = pipe.to(device)
```
it consumes about 75 seconds on my computer with A800 GPU.
But I found in comfyui, it only need 22 seconds to load flux model, but it load the fp8 model.
Can diffusers load flux fp8 model ?
or is there any other speed up method ?
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start with the FluxPipeline.from_pretrained example and its torch_dtype=torch.float16 loading path, then investigate the fp8 model behavior referenced from ComfyUI. Determine whether diffusers can load and run FLUX with fp8 or provide another reproducible loading speed-up for the stated A800 scenario. Done means the supported approach and its measured result are documented or validated.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python, pytorch
- Ambito
- machine-learning
- Tipo di issue
- Funzionalità
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 30/100