huggingface / huggingface/diffusers

Model and input data type is not same

Abierto
#8,780 1 comentario 0 reacciones 0 asignados Ver en GitHub
stale
Lenguaje dominante
Python
Estrellas
34.5k
Forks
7.3k
Merge medio
3 d 3 h
PR fusionados (30 d)
91

Descripción

**Is your feature request related to a problem? Please describe.**
Hi, when I trained sdv1.5 model with fp16 mode by using the `examples/text_to_image/train_text_to_image.py` file, I found there is a mismatch between unet model and input data. Specificaly, In this [line](https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image.py#L993) , the `unet` model has float32 dtype, but the `noisy_latents` has the float16 dtype. Although it will not raise an error in cuda , I use my custom device it will raise an error, I wonder how can I change this code to use float16.

**Describe the solution you'd like.**
To avoid get a wrong model, I would like you give a right code to match model and input.

**Describe alternatives you've considered.**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context.**
Add any other context or screenshots about the feature request here.

Guía de contribución

Abrir la guía de contribución

Línea de trabajo

Read examples/text_to_image/train_text_to_image.py around line 993, focusing on the dtype of the unet model and noisy_latents in fp16 mode. Reproduce the mismatch on the reported custom device and verify that the training path passes matching dtypes without breaking the existing CUDA behavior.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
python, pytorch
Área
machine-learning
Tipo de issue
Error
Dificultad
2/5
Tiempo estimado
1-3 horas
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
48/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.