huggingface / huggingface/diffusers

UNet-only adapters will still cause problems with pipeline.set_adapters() after torch.compile because the compiled graph expects adapters to exist in all components.

Abierto
#12,427 3 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

stale
Lenguaje dominante
Python
Estrellas
34.5k
Forks
7.3k
Merge medio
3 d 3 h
PR fusionados (30 d)
91

Descripción

2025-10-03 07:47:56 - main - INFO - loading adapter detail_enhancer
No LoRA keys associated to CLIPTextModel found with the prefix='text_encoder'. This is safe to ignore if LoRA state dict didn't originally have any CLIPTextModel related params. You can also try specifying prefix=None to resolve the warning. Otherwise, open an issue if you think it's unexpected: https://github.com/huggingface/diffusers/issues/new
No LoRA keys associated to CLIPTextModelWithProjection found with the prefix='text_encoder_2'. This is safe to ignore if LoRA state dict didn't originally have any CLIPTextModelWithProjection related params. You can also try specifying prefix=None to resolve the warning. Otherwise, open an issue if you think it's unexpected: https://github.com/huggingface/diffusers/issues/new

This warning confirms what we've been discussing: detail_enhancer (and likely nijistyle and chibi_rr) only have UNet weights, not text encoder weights.

The warnings are saying the LoRA files don't contain any parameters for the text encoders - they're UNet-only adapters.
Why This Causes Issues After Compilation

When you call pipeline.set_adapters(['detail_enhancer']):

Without compilation:

Pipeline checks each component
Applies adapter to UNet ✓
Skips text encoders (no weights found) ✓
Works fine

With compilation:

The compiled graph expects a consistent structure
If you compile after loading adapters that ARE in text encoders, the graph expects text encoder adapters
When you try to use detail_enhancer which has no text encoder weights, the compiled code path fails

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Línea de trabajo

Empieza reproduciendo la interacción entre pipeline.set_adapters() y torch.compile descrita en la issue, comparando adaptadores con y sin pesos del codificador de texto. Lee los puntos de entrada para la carga de adaptadores y para la pipeline compilada para determinar el comportamiento esperado y define una prueba de regresión para adaptadores UNet-only.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
python, pytorch
Área
machine-learning
Tipo de issue
Error
Dificultad
5/5
Tiempo estimado
Más de una semana
Estado de actividad
Estancado
Claridad
Necesita aclaración
Aptitud para principiantes
30/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.