huggingface / huggingface/lora-fast

Generate Different Number of Images Without Recompilation

Open
#5 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
32
Forks
2
PR merge metrics
No merged PRs in 30d

Description

I'm trying to generate a different number of images without recompilation. For example, warm up with 1 image and then generate with 2 images. I tried with dynamic=True, but I still get recompilation when changing num_images_per_prompt:

```python
with torch._dynamo.config.patch(error_on_recompile=True):
pipe = DiffusionPipeline.from_pretrained(
"black-forest-labs/FLUX.1-dev,
torch_dtype=torch.bfloat16,
).to("cuda")

pipe.transformer = torch.compile(pipe.transformer, dynamic=True)

# warmup
image = pipe(prompt="A cat holding a sign that says hello world",
width=1024,
height=1024,
num_inference_steps=10,
num_images_per_prompt=1,
).images[0]

# inference
image = pipe(prompt="A cat holding a sign that says hello world",
width=1024,
height=1024,
num_inference_steps=10,
num_images_per_prompt=2,
).images[0]
```

The error:
```
torch._dynamo.exc.RecompileError: Recompiling function forward in /root/PycharmProjects/project/.venv/lib/python3.10/site-packages/diffusers/models/transformers/transformer_flux.py:389
triggered by the following guard failure(s):
- 0/0: tensor 'guidance' size mismatch at index 0. expected 1, actual 2
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.