kohya-ss / kohya-ss/sd-scripts

how to train stabilityai/stable-diffusion-2-inpainting via lora trainning?

Open
#170 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.2k
Forks
1.2k
Avg merge
11m
Merged PRs (30d)
2

Description

as title,
![img_v2_047d47ce-fbd3-4b72-ac2b-a7a2137154ag](https://user-images.githubusercontent.com/55580043/217997379-9806c09b-ad27-4aec-8b57-993f86453520.jpg)
![img_v2_fbe67f30-f95a-4b9d-bd40-776524b7761g](https://user-images.githubusercontent.com/55580043/217997403-4d72b603-ca18-41b1-b5e9-1ac6605f30ab.jpg)
![img_v2_281ee42a-f2c2-4367-8925-f2298aa66f8g](https://user-images.githubusercontent.com/55580043/217997416-aae3ab99-7df9-4277-a681-a0a01a0fd560.jpg)

the error is :
Traceback (most recent call last):
File "/ai/home/project/sd-scripts-main/train_network.py", line 538, in
train(args)
File "/ai/home/project/sd-scripts-main/train_network.py", line 408, in train
noise_pred = unet(noisy_latents, timesteps, encoder_hidden_states).sample
File "/ai/home/project/sd-scripts-main/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/ai/home/project/sd-scripts-main/venv/lib/python3.10/site-packages/accelerate/utils/operations.py", line 490, in __call__
return convert_to_fp32(self.model_forward(*args, **kwargs))
File "/ai/home/project/sd-scripts-main/venv/lib/python3.10/site-packages/torch/amp/autocast_mode.py", line 12, in decorate_autocast
return func(*args, **kwargs)
File "/ai/home/project/sd-scripts-main/venv/lib/python3.10/site-packages/diffusers/models/unet_2d_condition.py", line 375, in forward
sample = self.conv_in(sample)
File "/ai/home/project/sd-scripts-main/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/ai/home/project/sd-scripts-main/venv/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 457, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/ai/home/project/sd-scripts-main/venv/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 453, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [320, 9, 3, 3], expected input[2, 4, 64, 64] to have 9 channels, but got 4 channels instead

so how can i train that stabilityai/stable-diffusion-2-inpainting?

Contributor guide

No contributing guide indexed for this repository

Research direction

The traceback points to train_network.py and the diffusers UNet forward path; start in train() around the reported line and trace how the model input is assembled. Compare the supplied four-channel input with the UNet's nine-channel expectation, then determine whether inpainting training is supported by this entry point and what a complete supported workflow would require.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
ai, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.