huggingface / huggingface/diffusers

Improve SD35 LoRA support to cover most popular LoRA formats

Offen
#9,950 9 Kommentare 1 Reaktion 1 zugewiesene Person Beansprucht von @sayakpaul Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
34.5k
Forks
7.3k
Ø Merge
3 T. 3 Std.
Gemergte PRs (30 T.)
91

Beschreibung

SD3.x pipeline does implement `SD3LoraLoaderMixin` and as such `load_lora_weights` on SD3.x does "work".

However, attempting to load any of the most popular LoRAs results in silent failure:
load is successful without any warnings, but loads ZERO keys.

Looking at implementation at:

Shows that if there are no matching keys in `text_encoder_state_dict`, `text_encoder_state_dict`, `text_encoder_2_state_dict`, there is no warning raised, the method simply does NOTHING.

When looking at `state_dict` of loaded LoRA, it shows that keys are NOT in the expected format - they need remapping.
For example:
```dict_keys(['lora_unet_joint_blocks_0_context_block_adaLN_modulation_1.alpha', 'lora_unet_joint_blocks_0_context_block_adaLN_modulation_1.lora_down.weight', 'lora_unet_joint_blocks_0_context_block_adaLN_modulation_1.lora_up.weight', 'lora_unet_joint_blocks_0_context_block_attn_proj.alpha' ...])```

Below are download links for some of the popular LoRAs for SD35, all experience same behavior:
-
-
-

CC @yiyixuxu @sayakpaul @DN6 @asomoza
CC @AI-Casanova

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.