lllyasviel / lllyasviel/stable-diffusion-webui-forge
ControlNet and Kohya HRFix interop support
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
Is there an existing issue for this?
- I have searched the existing issues and checked the recent builds/commits
What would your feature do ?
我经常使用高分辨率的图生图,会使用CN的lineArt和Tile两个模型的样子。而且我发现,小分辨率的时候(1000px),SD生成的图,光影和结构更加正确、优秀,超过2000px的时候就会变得很平、很容易出错的样子,无论是结构上,还是光影。
然后我使用了内置的Kohya HRFix,它的工作机制就是先缩放分辨率迭代出正确的结构,然后再提升会原本的分辨率,而且可以控制参与的步骤的样子。在文生图(1500x1500,甚至2000x2000以上),能得到结构正确且非常优秀的画面。
我就想着试着搭配CN模型一起,进一步控制画面的样子,然后就出现了报错的样子,在Kohya HRFix起作用的阶段中,CN就会出现
warning control could not be applied torch.Size([2, 1280, 11, 8]) torch.Size([2, 1280, 32, 22])
warning control could not be applied torch.Size([2, 1280, 11, 8]) torch.Size([2, 1280, 32, 22])
warning control could not be applied torch.Size([2, 1280, 11, 8]) torch.Size([2, 1280, 32, 22])
warning control could not be applied torch.Size([2, 1280, 22, 15]) torch.Size([2, 1280, 64, 43])
warning control could not be applied torch.Size([2, 1280, 22, 15]) torch.Size([2, 1280, 64, 43])
warning control could not be applied torch.Size([2, 640, 22, 15]) torch.Size([2, 640, 64, 43])
warning control could not be applied torch.Size([2, 640, 43, 29]) torch.Size([2, 640, 128, 86])
warning control could not be applied torch.Size([2, 640, 43, 29]) torch.Size([2, 640, 128, 86])
然后并不起作用的样子。因为这个分辨率和CN设定的原本的分辨率不匹配的样子。
所以我特别希望CN也能自动匹配和Kohya HRFix的流程。
Proposed workflow
- 到图生图/文生图,写好参数和设置
- 启用Kohya HRFix Integrated,设置好参数
- 启用CN,设置好参数和模型
- CN能和HRFix一起正常运转
Additional information
No response
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the documented text-to-image or image-to-image workflow with Integrated Kohya HRFix and ControlNet, then inspect the ControlNet integration at the HRFix stage where the reported tensor-shape warnings occur. Done means ControlNet applies during HRFix without those warnings and preserves the configured conditioning.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- machine-learning, python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100