huggingface / huggingface/diffusers

Model and input data type is not same

Ouverte
#8,780 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub
stale
Langage dominant
Python
Étoiles
34.5k
Forks
7.3k
Merge moyen
3 j 3 h
PR mergées (30 j)
91

Description

**Is your feature request related to a problem? Please describe.**
Hi, when I trained sdv1.5 model with fp16 mode by using the `examples/text_to_image/train_text_to_image.py` file, I found there is a mismatch between unet model and input data. Specificaly, In this [line](https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image.py#L993) , the `unet` model has float32 dtype, but the `noisy_latents` has the float16 dtype. Although it will not raise an error in cuda , I use my custom device it will raise an error, I wonder how can I change this code to use float16.

**Describe the solution you'd like.**
To avoid get a wrong model, I would like you give a right code to match model and input.

**Describe alternatives you've considered.**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context.**
Add any other context or screenshots about the feature request here.

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Read examples/text_to_image/train_text_to_image.py around line 993, focusing on the dtype of the unet model and noisy_latents in fp16 mode. Reproduce the mismatch on the reported custom device and verify that the training path passes matching dtypes without breaking the existing CUDA behavior.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
python, pytorch
Domaine
machine-learning
Type d'issue
Bug
Difficulté
2/5
Temps estimé
1-3 heures
Activité
À l'abandon
Clarté
Plutôt claire
Accessibilité débutants
48/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.