huggingface / huggingface/diffusers

When use UniPCMultistepScheduler as the ODE solver, and input different num_inference_steps in StableDiffusionXLPipeline, the first inference step is wrong

Offen
#13,887 3 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug needs-code-example pipelines schedulers
Vorherrschende Sprache
Python
Sterne
34.5k
Forks
7.3k
Ø Merge
3 T. 3 Std.
Gemergte PRs (30 T.)
91

Beschreibung

### Describe the bug

When the user calls the StableDiffusionXLPipeline using the ckpt from stabilityai/stable-diffusion-xl-base-1.0 and sets UniPCMultistepScheduler as the inference schedule, if the user only inputs num_inference_steps without setting timesteps, the inference time schedule would be incorrect. Setting different num_inference_steps will lead to different first step input time labels, even if in all of the situations, the first step input x is pure Gaussian noise.

### Reproduction

DTYPE = torch.float16 # torch.float16 works as well, but pictures seem to be a bit worse
device = "cuda"

pipe = StableDiffusionXLPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0",torch_dtype=DTYPE)
scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config)
pipe.scheduler = scheduler
pipe.to(device)
pipe.scheduler.set_timesteps(10)
print(pipe.scheduler.timesteps)
pipe.scheduler.set_timesteps(15)
print(pipe.scheduler.timesteps)
""""
In diffusers\src\diffusers\pipelines\stable_diffusion_xl\pipeline_stable_diffusion_xl.py,
Once the timestep been set by retrieve_timesteps in line 1103
, the inference timesteps are not updated to add more elements
, then, in the line 1193 the inference loop start
, and the u_net start feedforward in line 1212.
The u_net will input different first timesteps when the user set different inference nums.
"""

### Logs

```shell
The output log of the code in the reproduction section is:
tensor([901, 811, 721, 631, 541, 451, 361, 271, 181, 91])
tensor([931, 869, 807, 745, 683, 621, 559, 497, 435, 373, 311, 249, 187, 125, 63])
```

### System Info

- 🤗 Diffusers version: 0.38.0
- Platform: Windows-10-10.0.26200-SP0
- Running on Google Colab?: No
- Python version: 3.11.15
- PyTorch version (GPU?): 2.12.0+cu132 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Huggingface_hub version: 1.18.0
- Transformers version: 5.10.2
- Accelerate version: not installed
- PEFT version: not installed
- Bitsandbytes version: not installed
- Safetensors version: 0.8.0-rc.1
- xFormers version: not installed
- Accelerator: NVIDIA GeForce RTX 5060 Ti, 16311 MiB
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help?

@yiyixuxu @sayakpaul @DN6

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Reproduzieren Sie das Problem mit StableDiffusionXLPipeline und UniPCMultistepScheduler und lesen Sie in diffusers/src/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py rund um retrieve_timesteps in Zeile 1103, die Inferenzschleife in Zeile 1193 und den U-Net-Aufruf in Zeile 1212. Vergleichen Sie die Schedules und den ersten Eingabe-Timestep für unterschiedliche num_inference_steps; abgeschlossen bedeutet, dass das Verhalten des Schedules im ersten Schritt korrekt ist, wenn nur dieser Parameter angegeben wird.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python, pytorch
Bereich
machine-learning
Issue-Typ
Bug
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Ruhig
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
48/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.