kohya-ss / kohya-ss/sd-scripts
merge SD2.1 Lora models error
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 1.2k
- Avg merge
- 11m
- Merged PRs (30d)
- 2
Description
I want to merge SD2.1 512 with three loras trained on it
using --v2 argument gives this error :
```
Traceback (most recent call last):
File "C:\Users\Axel\sd-scripts\networks\merge_lora.py", line 243, in
merge(args)
File "C:\Users\Axel\sd-scripts\networks\merge_lora.py", line 194, in merge
merge_to_sd_model(text_encoder, unet, args.models, args.ratios, merge_dtype)
File "C:\Users\Axel\sd-scripts\networks\merge_lora.py", line 98, in merge_to_sd_model
conved = torch.nn.functional.conv2d(down_weight.permute(1, 0, 2, 3), up_weight).permute(1, 0, 2, 3)
RuntimeError: number of dims don't match in permute
```
for this command
```PS \sd-scripts> py "networks\merge_lora.py" --sd_model "Documents/webui/stable-diffusion-webui/models/Stable-diffusion/v2-1_512-ema-pruned.safetensors" --save_precision fp16 --precision float --save_to "webui/stable-diffusion-webui/models/Stable-diffusion/newmodel.safetensors" --models "webui/stable-diffusion-webui/models/Lora/pixel3D_v1-000020.safetensors" "webui/stable-diffusion-webui/models/Lora/pixel3D-000030.safetensors" "webui/stable-diffusion-webui/models/Lora/pixel3D_v2-000040.safetensors" --ratios 0.22 0.22 0.22```
I have this error:
```
(venv) PS \sd-scripts> py "networks\merge_lora.py" --sd_model "Documents/webui/stable-diffusion-webui/models/Stable-diffusion/v2-1_512-ema-pruned.safetensors" --save_precision fp16 --precision float --save_to "webui/stable-diffusion-webui/models/Stable-diffusion/newmodel.safetensors" --models "webui/stable-diffusion-webui/models/Lora/pixel3D_v1-000020.safetensors" "webui/stable-diffusion-webui/models/Lora/pixel3D-000030.safetensors" "webui/stable-diffusion-webui/models/Lora/pixel3D_v2-000040.safetensors" --ratios 0.22 0.22 0.22
loading SD model: webui/stable-diffusion-webui/models/Stable-diffusion/v2-1_512-ema-pruned.safetensors
Traceback (most recent call last):
File "sd-scripts\networks\merge_lora.py", line 243, in
merge(args)
File "sd-scripts\networks\merge_lora.py", line 192, in merge
text_encoder, vae, unet = model_util.load_models_from_stable_diffusion_checkpoint(args.v2, args.sd_model)
File "sd-scripts\venv\lib\site-packages\library\model_util.py", line 868, in load_models_from_stable_diffusion_checkpoint
info = unet.load_state_dict(converted_unet_checkpoint)
File "sd-scripts\venv\lib\site-packages\torch\nn\modules\module.py", line 1604, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for UNet2DConditionModel:
size mismatch for down_blocks.0.attentions.0.proj_in.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with networks/merge_lora.py, especially merge() and merge_to_sd_model(), then inspect library/model_util.py at load_models_from_stable_diffusion_checkpoint. Reproduce the supplied command and compare the checkpoint tensor shapes with the model shapes shown in the traceback. Done means SD2.1 LoRA models load and merge without either reported error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100