huggingface / huggingface/diffusers
Improve SD35 LoRA support to cover most popular LoRA formats
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
SD3.x pipeline does implement `SD3LoraLoaderMixin` and as such `load_lora_weights` on SD3.x does "work".
However, attempting to load any of the most popular LoRAs results in silent failure:
load is successful without any warnings, but loads ZERO keys.
Looking at implementation at:
Shows that if there are no matching keys in `text_encoder_state_dict`, `text_encoder_state_dict`, `text_encoder_2_state_dict`, there is no warning raised, the method simply does NOTHING.
When looking at `state_dict` of loaded LoRA, it shows that keys are NOT in the expected format - they need remapping.
For example:
```dict_keys(['lora_unet_joint_blocks_0_context_block_adaLN_modulation_1.alpha', 'lora_unet_joint_blocks_0_context_block_adaLN_modulation_1.lora_down.weight', 'lora_unet_joint_blocks_0_context_block_adaLN_modulation_1.lora_up.weight', 'lora_unet_joint_blocks_0_context_block_attn_proj.alpha' ...])```
Below are download links for some of the popular LoRAs for SD35, all experience same behavior:
-
-
-
CC @yiyixuxu @sayakpaul @DN6 @asomoza
CC @AI-Casanova
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.