huggingface / huggingface/diffusers

Expanded init fields in StableDiffusionPipeline cause incompatibilities with many/most inherited pipelines

Offen
#6,969 22 Kommentare 0 Reaktionen 1 zugewiesene Person Beansprucht von @asomoza Auf GitHub ansehen
bug contributions-welcome good first issue help wanted wip
Vorherrschende Sprache
Python
Sterne
34.5k
Forks
7.3k
Ø Merge
3 T. 3 Std.
Gemergte PRs (30 T.)
91

Beschreibung

### Describe the bug

class `StableDiffusionPipeline` has its init section as:
```py
def __init__(
self,
vae: AutoencoderKL,
text_encoder: CLIPTextModel,
tokenizer: CLIPTokenizer,
unet: UNet2DConditionModel,
scheduler: KarrasDiffusionSchedulers,
safety_checker: StableDiffusionSafetyChecker,
feature_extractor: CLIPImageProcessor,
image_encoder: CLIPVisionModelWithProjection = None,
requires_safety_checker: bool = True,
):
```

and here `image_encoder` was recently introduced thus changing the class signature
but most community pipelines do not include init field for recently included `image_encoder` and thus order or params is wrong

for example, `examples/community/regional_prompting_stable_diffusion.py` has this in its init:
```py
super().__init__(
vae,
text_encoder,
tokenizer,
unet,
scheduler,
safety_checker,
feature_extractor,
requires_safety_checker,
)
```
which means `bool` value from `requires_safety_checker` is going to be passed as `image_encoder` and pipeline will fail during initialization like this:

```log
> diffusers/pipelines/pipeline_utils.py:546 in _fetch_class_library_tuple
AttributeError: 'bool' object has no attribute '__module__'
```

this is a conceptual problem with changing master class signature while all inherited classes pass args list as simple list.
i don't see a simple solution as going back is bad and going forward requires updates to a lot of pipelines.

but at the very basic, at least add error handling to `_fetch_class_library_tuple` so invalid type does not cause entire solution to crash.

### Reproduction

load nearly any community pipeline or any pipeline inherited from StableDiffusionPipeline, but not updated to use new signature in its `super().__init__` call

### Logs

_No response_

### System Info

diffusers==0.26.3

### Who can help?

@yiyixuxu @DN6 @sayakpaul @patrickvonplaten

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.