huggingface / huggingface/diffusers

Add multi aspect ratio, and multi crop SCM training script

オープン
#5,876 コメント 11 件 リアクション 0 件 担当者 1 名 @patil-suraj が担当を希望しています GitHub で見る
stale
主要言語
Python
スター
34.5k
フォーク
7.3k
平均マージ
3日 3時間
マージ済み PR(30日)
91

説明

None of the diffusers training scripts support multiple crops or aspect ratios in the data loader. This is mentioned in the tech report for SDXL as a part of their training process.

In my tests diffusion model struggle to do high resolution details. I did this for base SD 1.5 and was planning to port the code to the SDXL SCM scripts.

@sayakpaul I know you wanted to keep the base training scripts simple. Wanted to open a discussion here on whether to upstream. Is this something I should upstream? or better as something like a community script?

The SCM models are fantastic, but full short of the base SDXL model on high resolution details. Hoping this can fix some of that up.

References
------------
SCM vs LDMs inference report ( Need to add multiple aspect ratios to inference report)
https://wandb.ai/nbardy-facet/LCM-Inference-Test/reports/SDXL--Vmlldzo2MDAzNDU1

SCM LORA 4 steps
image
Base SDXL 30 steps
image

SDXL technical report aspect ratios.

image

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。