huggingface / huggingface/diffusers

Add multi aspect ratio, and multi crop SCM training script

Aperta
#5,876 11 commenti 0 reazioni 1 assegnatario Rivendicata da @patil-suraj Vedi su GitHub
stale
Lingua principale
Python
Stelle
34.5k
Fork
7.3k
Merge medio
3g 3h
PR unite (30g)
91

Descrizione

None of the diffusers training scripts support multiple crops or aspect ratios in the data loader. This is mentioned in the tech report for SDXL as a part of their training process.

In my tests diffusion model struggle to do high resolution details. I did this for base SD 1.5 and was planning to port the code to the SDXL SCM scripts.

@sayakpaul I know you wanted to keep the base training scripts simple. Wanted to open a discussion here on whether to upstream. Is this something I should upstream? or better as something like a community script?

The SCM models are fantastic, but full short of the base SDXL model on high resolution details. Hoping this can fix some of that up.

References
------------
SCM vs LDMs inference report ( Need to add multiple aspect ratios to inference report)
https://wandb.ai/nbardy-facet/LCM-Inference-Test/reports/SDXL--Vmlldzo2MDAzNDU1

SCM LORA 4 steps
image
Base SDXL 30 steps
image

SDXL technical report aspect ratios.

image

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.