lllyasviel / lllyasviel/ControlNet
Issue with SD 2.1: size mismatch for input_blocks.1.1.proj_in.weight
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 34.1k
- Forks
- 3k
- PR merge metrics
- No merged PRs in 30d
Description
After upgrading to the latest ControlNet I run into a size mismatch exception. Any ideas what could be causing it?
Control net works fine with SD 1.5
Here's the error log:
Loading model: control_v11p_sd21_depth [2722c7d7]
Loaded state_dict from [D:\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_v11p_sd21_depth.safetensors]
Loading config: D:\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_v11p_sd21_depth.yaml
Error running process: D:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
Traceback (most recent call last):
File "D:\stable-diffusion-webui\modules\scripts.py", line 417, in process
script.process(p, *script_args)
File "D:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 959, in process
model_net = self.load_control_model(p, unet, unit.model, unit.low_vram)
File "D:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 645, in load_control_model
model_net = self.build_control_model(p, unet, model, lowvram)
File "D:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 713, in build_control_model
network = network_module(
File "D:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\cldm.py", line 107, in __init__
self.control_model.load_state_dict(state_dict)
File "D:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1671, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for ControlNet:
size mismatch for input_blocks.1.1.proj_in.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
size mismatch for input_blocks.1.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([320, 768]).
size mismatch for input_blocks.1.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([320, 768]).
size mismatch for input_blocks.1.1.proj_out.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
size mismatch for input_blocks.2.1.proj_in.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
size mismatch for input_blocks.2.1.transformer_blocks.0.attn2.to_k.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([320, 768]).
size mismatch for input_blocks.2.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([320, 1024]) from checkpoint, the shape in current model is torch.Size([320, 768]).
size mismatch for input_blocks.2.1.proj_out.weight: copying a param with shape torch.Size([320, 320]) from checkpoint, the shape in current model is torch.Size([320, 320, 1, 1]).
....
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in scripts/controlnet.py at build_control_model and scripts/cldm.py at ControlNet.load_state_dict. Reproduce the failure with control_v11p_sd21_depth.safetensors and its YAML config, then compare the checkpoint tensor shapes with the constructed model and the SD 1.5 path. Done means identifying the SD 2.1 compatibility mismatch and documenting or correcting the responsible configuration or loading behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100