huggingface / huggingface/diffusers

Expanded init fields in StableDiffusionPipeline cause incompatibilities with many/most inherited pipelines

Ouverte
#6,969 22 commentaires 0 réactions 1 personne assignée Réclamée par @asomoza Voir sur GitHub
bug contributions-welcome good first issue help wanted wip
Langage dominant
Python
Étoiles
34.5k
Forks
7.3k
Merge moyen
3 j 3 h
PR mergées (30 j)
91

Description

### Describe the bug

class `StableDiffusionPipeline` has its init section as:
```py
def __init__(
self,
vae: AutoencoderKL,
text_encoder: CLIPTextModel,
tokenizer: CLIPTokenizer,
unet: UNet2DConditionModel,
scheduler: KarrasDiffusionSchedulers,
safety_checker: StableDiffusionSafetyChecker,
feature_extractor: CLIPImageProcessor,
image_encoder: CLIPVisionModelWithProjection = None,
requires_safety_checker: bool = True,
):
```

and here `image_encoder` was recently introduced thus changing the class signature
but most community pipelines do not include init field for recently included `image_encoder` and thus order or params is wrong

for example, `examples/community/regional_prompting_stable_diffusion.py` has this in its init:
```py
super().__init__(
vae,
text_encoder,
tokenizer,
unet,
scheduler,
safety_checker,
feature_extractor,
requires_safety_checker,
)
```
which means `bool` value from `requires_safety_checker` is going to be passed as `image_encoder` and pipeline will fail during initialization like this:

```log
> diffusers/pipelines/pipeline_utils.py:546 in _fetch_class_library_tuple
AttributeError: 'bool' object has no attribute '__module__'
```

this is a conceptual problem with changing master class signature while all inherited classes pass args list as simple list.
i don't see a simple solution as going back is bad and going forward requires updates to a lot of pipelines.

but at the very basic, at least add error handling to `_fetch_class_library_tuple` so invalid type does not cause entire solution to crash.

### Reproduction

load nearly any community pipeline or any pipeline inherited from StableDiffusionPipeline, but not updated to use new signature in its `super().__init__` call

### Logs

_No response_

### System Info

diffusers==0.26.3

### Who can help?

@yiyixuxu @DN6 @sayakpaul @patrickvonplaten

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.