huggingface / huggingface/diffusers
Model and input data type is not same
- 主要言語
- Python
- スター
- 34.5k
- フォーク
- 7.3k
- 平均マージ
- 3日 3時間
- マージ済み PR(30日)
- 91
説明
**Is your feature request related to a problem? Please describe.**
Hi, when I trained sdv1.5 model with fp16 mode by using the `examples/text_to_image/train_text_to_image.py` file, I found there is a mismatch between unet model and input data. Specificaly, In this [line](https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image.py#L993) , the `unet` model has float32 dtype, but the `noisy_latents` has the float16 dtype. Although it will not raise an error in cuda , I use my custom device it will raise an error, I wonder how can I change this code to use float16.
**Describe the solution you'd like.**
To avoid get a wrong model, I would like you give a right code to match model and input.
**Describe alternatives you've considered.**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context.**
Add any other context or screenshots about the feature request here.
コントリビューションガイド
調査の方向性
Read examples/text_to_image/train_text_to_image.py around line 993, focusing on the dtype of the unet model and noisy_latents in fp16 mode. Reproduce the mismatch on the reported custom device and verify that the training path passes matching dtypes without breaking the existing CUDA behavior.
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- python, pytorch
- 領域
- machine-learning
- issue の種類
- バグ
- 難易度
- 2/5
- 見積もり時間
- 1〜3時間
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 48/100