kohya-ss / kohya-ss/sd-scripts

merge SD2.1 Lora models error

Open
#745 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
7.2k
Forks
1.2k
Avg merge
11m
Merged PRs (30d)
2

Description

I want to merge SD2.1 512 with three loras trained on it

using --v2 argument gives this error :
```
Traceback (most recent call last):
File "C:\Users\Axel\sd-scripts\networks\merge_lora.py", line 243, in
merge(args)
File "C:\Users\Axel\sd-scripts\networks\merge_lora.py", line 194, in merge
merge_to_sd_model(text_encoder, unet, args.models, args.ratios, merge_dtype)
File "C:\Users\Axel\sd-scripts\networks\merge_lora.py", line 98, in merge_to_sd_model
conved = torch.nn.functional.conv2d(down_weight.permute(1, 0, 2, 3), up_weight).permute(1, 0, 2, 3)
RuntimeError: number of dims don't match in permute
```
for this command
```PS \sd-scripts> py "networks\merge_lora.py" --sd_model "Documents/webui/stable-diffusion-webui/models/Stable-diffusion/v2-1_512-ema-pruned.safetensors" --save_precision fp16 --precision float --save_to "webui/stable-diffusion-webui/models/Stable-diffusion/newmodel.safetensors" --models "webui/stable-diffusion-webui/models/Lora/pixel3D_v1-000020.safetensors" "webui/stable-diffusion-webui/models/Lora/pixel3D-000030.safetensors" "webui/stable-diffusion-webui/models/Lora/pixel3D_v2-000040.safetensors" --ratios 0.22 0.22 0.22```
I have this error:
```
(venv) PS \sd-scripts> py "networks\merge_lora.py" --sd_model "Documents/webui/stable-diffusion-webui/models/Stable-diffusion/v2-1_512-ema-pruned.safetensors" --save_precision fp16 --precision float --save_to "webui/stable-diffusion-webui/models/Stable-diffusion/newmodel.safetensors" --models "webui/stable-diffusion-webui/models/Lora/pixel3D_v1-000020.safetensors" "webui/stable-diffusion-webui/models/Lora/pixel3D-000030.safetensors" "webui/stable-diffusion-webui/models/Lora/pixel3D_v2-000040.safetensors" --ratios 0.22 0.22 0.22
loading SD model: webui/stable-diffusion-webui/models/Stable-diffusion/v2-1_512-ema-pruned.safetensors
Traceback (most recent call last):
File "sd-scripts\networks\merge_lora.py", line 243, in
merge(args)
File "sd-scripts\networks\merge_lora.py", line 192, in merge
text_encoder, vae, unet = model_util.load_models_from_stable_diffusion_checkpoint(args.v2, args.sd_model)
File "sd-scripts\venv\lib\site-packages\library\model_util.py", line 868, in load_models_from_stable_diffusion_checkpoint
info = unet.load_state_dict(converted_unet_checkpoint)
File "sd-scripts\venv\lib\site-packages\torch\nn\modules\module.py", line 1604, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for UNet2DConditionModel:
size mismatch for down_blocks.0.attentions.0.proj_in.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with networks/merge_lora.py, especially merge() and merge_to_sd_model(), then inspect library/model_util.py at load_models_from_stable_diffusion_checkpoint. Reproduce the supplied command and compare the checkpoint tensor shapes with the model shapes shown in the traceback. Done means SD2.1 LoRA models load and merge without either reported error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.