huggingface / huggingface/diffusers
When use UniPCMultistepScheduler as the ODE solver, and input different num_inference_steps in StableDiffusionXLPipeline, the first inference step is wrong
- Vorherrschende Sprache
- Python
- Sterne
- 34.5k
- Forks
- 7.3k
- Ø Merge
- 3 T. 3 Std.
- Gemergte PRs (30 T.)
- 91
Beschreibung
### Describe the bug
When the user calls the StableDiffusionXLPipeline using the ckpt from stabilityai/stable-diffusion-xl-base-1.0 and sets UniPCMultistepScheduler as the inference schedule, if the user only inputs num_inference_steps without setting timesteps, the inference time schedule would be incorrect. Setting different num_inference_steps will lead to different first step input time labels, even if in all of the situations, the first step input x is pure Gaussian noise.
### Reproduction
DTYPE = torch.float16 # torch.float16 works as well, but pictures seem to be a bit worse
device = "cuda"
pipe = StableDiffusionXLPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0",torch_dtype=DTYPE)
scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config)
pipe.scheduler = scheduler
pipe.to(device)
pipe.scheduler.set_timesteps(10)
print(pipe.scheduler.timesteps)
pipe.scheduler.set_timesteps(15)
print(pipe.scheduler.timesteps)
""""
In diffusers\src\diffusers\pipelines\stable_diffusion_xl\pipeline_stable_diffusion_xl.py,
Once the timestep been set by retrieve_timesteps in line 1103
, the inference timesteps are not updated to add more elements
, then, in the line 1193 the inference loop start
, and the u_net start feedforward in line 1212.
The u_net will input different first timesteps when the user set different inference nums.
"""
### Logs
```shell
The output log of the code in the reproduction section is:
tensor([901, 811, 721, 631, 541, 451, 361, 271, 181, 91])
tensor([931, 869, 807, 745, 683, 621, 559, 497, 435, 373, 311, 249, 187, 125, 63])
```
### System Info
- 🤗 Diffusers version: 0.38.0
- Platform: Windows-10-10.0.26200-SP0
- Running on Google Colab?: No
- Python version: 3.11.15
- PyTorch version (GPU?): 2.12.0+cu132 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Huggingface_hub version: 1.18.0
- Transformers version: 5.10.2
- Accelerate version: not installed
- PEFT version: not installed
- Bitsandbytes version: not installed
- Safetensors version: 0.8.0-rc.1
- xFormers version: not installed
- Accelerator: NVIDIA GeForce RTX 5060 Ti, 16311 MiB
- Using GPU in script?:
- Using distributed or parallel set-up in script?:
### Who can help?
@yiyixuxu @sayakpaul @DN6
Beitragsleitfaden
Rechercherichtung
Reproduzieren Sie das Problem mit StableDiffusionXLPipeline und UniPCMultistepScheduler und lesen Sie in diffusers/src/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py rund um retrieve_timesteps in Zeile 1103, die Inferenzschleife in Zeile 1193 und den U-Net-Aufruf in Zeile 1212. Vergleichen Sie die Schedules und den ersten Eingabe-Timestep für unterschiedliche num_inference_steps; abgeschlossen bedeutet, dass das Verhalten des Schedules im ersten Schritt korrekt ist, wenn nur dieser Parameter angegeben wird.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- python, pytorch
- Bereich
- machine-learning
- Issue-Typ
- Bug
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Aktivitätsstatus
- Ruhig
- Klarheit
- Größtenteils klar
- Anfängerfreundlichkeit
- 48/100