huggingface / huggingface/diffusers

Can not load fusionx_lora into original wan2.1-14b

Open
#11,849 1 comment 0 reactions 0 assignees View on GitHub
stale
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

hello, i am adding the fusionx_lora into original wan2.1-14b-i2v, my code is as follow:

> pipe = WanImageToVideoPipeline.from_pretrained(my_local_path + "Wan2.1-I2V-14B-480P-Diffusers", vae=vae, image_encoder=image_encoder, torch_dtype=torch.bfloat16)
> pipe.load_lora_weights(
> my_local_path + "Wan14BT2VFusioniX/FusionX_LoRa/Wan2.1_I2V_14B_FusionX_LoRA.safetensors"
> )

But i got some errors:

> File "/mmu_mllm_hdd_2/zuofei/infer_test/lora_infer_multi.py", line 60, in process_image
> pipe.load_lora_weights(
> File "/hetu_group/zuofei/env/wan_infer/lib/python3.12/site-packages/diffusers/loaders/lora_pipeline.py", line 4869, in load_lora_weights
> state_dict = self.lora_state_dict(pretrained_model_name_or_path_or_dict, **kwargs)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/hetu_group/zuofei/env/wan_infer/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
> return fn(*args, **kwargs)
> ^^^^^^^^^^^^^^^^^^^
> File "/hetu_group/zuofei/env/wan_infer/lib/python3.12/site-packages/diffusers/loaders/lora_pipeline.py", line 4796, in lora_state_dict
> state_dict = _convert_non_diffusers_wan_lora_to_diffusers(state_dict)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/hetu_group/zuofei/env/wan_infer/lib/python3.12/site-packages/diffusers/loaders/lora_conversion_utils.py", line 1564, in _convert_non_diffusers_wan_lora_to_diffusers
> num_blocks = len({k.split("blocks.")[1].split(".")[0] for k in original_state_dict})
> ~~~~~~~~~~~~~~~~~~^^^
> IndexError: list index out of range

Can you tell me how to fix it? Thank you so much!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.