huggingface / huggingface/diffusers

FLUX.1-dev FP8 Example Code Cleanup

Open
#9,690 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

I was looking at the [FLUX.1-dev FP8 example code](https://huggingface.co/docs/diffusers/main/en/api/pipelines/flux#single-file-loading-for-the-fluxtransformer2dmodel) in the documentation and noticed some unnecessary imports and variable declarations, which make it appear longer and cluttered. Here is the shorter version of the code I'm currently using. Please note that I'm not an expert, just trying to make it easier for anyone else looking to run the code.

```
import torch
from diffusers import FluxTransformer2DModel, FluxPipeline
from transformers import T5EncoderModel
from optimum.quanto import quantize, qfloat8, freeze

bfl_repo = "black-forest-labs/FLUX.1-dev"

transformer = FluxTransformer2DModel.from_single_file("https://huggingface.co/Kijai/flux-fp8/blob/main/flux1-dev-fp8.safetensors", torch_dtype=torch.bfloat16)
quantize(transformer, weights=qfloat8)
freeze(transformer)

text_encoder_2 = T5EncoderModel.from_pretrained(bfl_repo, subfolder="text_encoder_2", torch_dtype=torch.bfloat16)
quantize(text_encoder_2, weights=qfloat8)
freeze(text_encoder_2)

pipe = FluxPipeline.from_pretrained(bfl_repo, transformer=transformer, text_encoder_2=text_encoder_2, torch_dtype=torch.bfloat16)
pipe.enable_model_cpu_offload()

prompt = "A cat holding a sign that says hello world"
image = pipe(
prompt,
generator=torch.Generator("cpu").manual_seed(0)
).images[0]

image.save("flux-dev-fp8.png")
```

Optimizations made:
Removed unnecessary code and reduced the line count from 32 to 26.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.