huggingface / huggingface/diffusers

Add multi aspect ratio, and multi crop SCM training script

Abierto
#5,876 11 comentarios 0 reacciones 1 asignado Reclamado por @patil-suraj Ver en GitHub
stale
Lenguaje dominante
Python
Estrellas
34.5k
Forks
7.3k
Merge medio
3 d 3 h
PR fusionados (30 d)
91

Descripción

None of the diffusers training scripts support multiple crops or aspect ratios in the data loader. This is mentioned in the tech report for SDXL as a part of their training process.

In my tests diffusion model struggle to do high resolution details. I did this for base SD 1.5 and was planning to port the code to the SDXL SCM scripts.

@sayakpaul I know you wanted to keep the base training scripts simple. Wanted to open a discussion here on whether to upstream. Is this something I should upstream? or better as something like a community script?

The SCM models are fantastic, but full short of the base SDXL model on high resolution details. Hoping this can fix some of that up.

References
------------
SCM vs LDMs inference report ( Need to add multiple aspect ratios to inference report)
https://wandb.ai/nbardy-facet/LCM-Inference-Test/reports/SDXL--Vmlldzo2MDAzNDU1

SCM LORA 4 steps
image
Base SDXL 30 steps
image

SDXL technical report aspect ratios.

image

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.