huggingface / huggingface/diffusers

Expanded init fields in StableDiffusionPipeline cause incompatibilities with many/most inherited pipelines

Abierto
#6,969 22 comentarios 0 reacciones 1 asignado Reclamado por @asomoza Ver en GitHub
bug contributions-welcome good first issue help wanted wip
Lenguaje dominante
Python
Estrellas
34.5k
Forks
7.3k
Merge medio
3 d 3 h
PR fusionados (30 d)
91

Descripción

### Describe the bug

class `StableDiffusionPipeline` has its init section as:
```py
def __init__(
self,
vae: AutoencoderKL,
text_encoder: CLIPTextModel,
tokenizer: CLIPTokenizer,
unet: UNet2DConditionModel,
scheduler: KarrasDiffusionSchedulers,
safety_checker: StableDiffusionSafetyChecker,
feature_extractor: CLIPImageProcessor,
image_encoder: CLIPVisionModelWithProjection = None,
requires_safety_checker: bool = True,
):
```

and here `image_encoder` was recently introduced thus changing the class signature
but most community pipelines do not include init field for recently included `image_encoder` and thus order or params is wrong

for example, `examples/community/regional_prompting_stable_diffusion.py` has this in its init:
```py
super().__init__(
vae,
text_encoder,
tokenizer,
unet,
scheduler,
safety_checker,
feature_extractor,
requires_safety_checker,
)
```
which means `bool` value from `requires_safety_checker` is going to be passed as `image_encoder` and pipeline will fail during initialization like this:

```log
> diffusers/pipelines/pipeline_utils.py:546 in _fetch_class_library_tuple
AttributeError: 'bool' object has no attribute '__module__'
```

this is a conceptual problem with changing master class signature while all inherited classes pass args list as simple list.
i don't see a simple solution as going back is bad and going forward requires updates to a lot of pipelines.

but at the very basic, at least add error handling to `_fetch_class_library_tuple` so invalid type does not cause entire solution to crash.

### Reproduction

load nearly any community pipeline or any pipeline inherited from StableDiffusionPipeline, but not updated to use new signature in its `super().__init__` call

### Logs

_No response_

### System Info

diffusers==0.26.3

### Who can help?

@yiyixuxu @DN6 @sayakpaul @patrickvonplaten

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.