lllyasviel / lllyasviel/stable-diffusion-webui-forge
[Bug]: Multidiffusion integrated + controlnet integrated errors
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
### Checklist
- [ ] The issue exists after disabling all extensions
- [X] The issue exists on a clean installation of webui
- [X] The issue is caused by an extension, but I believe it is caused by a bug in the webui
- [X] The issue exists in the current version of the webui
- [X] The issue has not been reported before recently
- [ ] The issue has been reported before but has not been fixed yet
### What happened?
I checked multiple possible combinations of Multidiffusion Integrated + Controlnet Integrated (tile model) and there are some combinations that fail with errors.
Lets start with what works:
1. Multidiffusion Integrated works well without controlnet with both SD1.5 and SDXL
2. MultiDiffusion (method name) + SD1.5 + controlnet tile_resample - works well
However other combinations cause problems, from what I checked:
1. Mixture of Diffusers + SD1.5 + controlnet any preprocessor = "error_1"
2. MultiDiffusion + SD1.5 + controlnet tile_colorfix+sharp = "error_2"
3. MultiDiffusion/Mixture of Diffusers + SDXL + controlnet any preprocessor = "error_3"
I'm adding console logs as files to this issue (in "Additional information" section).
In the extension for normal webui (https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111) problematic scenarios 1 and 2 worked fine - multidiffusion extension worked on SD1.5 with all combinations of controlnet
For SDXL controlnet tile I used these:
https://huggingface.co/bdsqlsz/qinglong_controlnet-lllite
models named "bdsqlsz_controlllite_xl_tile_anime_X" - they seem to work fine in Ultimate SD Upscale extension
For SD1.5 I used the standard controlnet tile model "control_v11f1e_sd15_tile"
In Multidiffusion Integrated for SDXL I set tile size = 1024, other settings were left at defaults. For SD1.5 all settings were left at defaults.
All tests were made in img2img tab with processing image upscaled in extras with "resize by" = 1, image size was 2304x3456
There were some other issues made for multidiffusion extensions but none seems to describe current situation
Can we expect support for mentioned scenarios (at least for those that worked in original extension)?
Also I noticed that slider "Tile Batch Size" in Multidiffusion Integrated has allowed range of values from 1 to 8192, this looks like a much too big number for batch size. Or am I missing something and this is intentional?
If it matters I installed webui-forge from the "one-click installation package" in readme file (I also ran update.bat so forge version should be up to date)
### Steps to reproduce the problem
1. Go to img2img
2. Add input image
3. set "resize by" to 1 (this shouldn't matter actually, I just did testing with this number)
4. enable Controlnet Integrated with tile model (preprocessors described in "What happened?" section)
5. enable Multidiffusion Integrated and choose method as described in "What happened?" section
6. generate image
### What should have happened?
Process the image with Multidiffusion Integrated + Controlnet Integrated (tile model) without errors and produce output image
### What browsers do you use to access the UI ?
Brave
### Sysinfo
[sysinfo-2024-02-11-12-43.json](https://github.com/lllyasviel/stable-diffusion-webui-forge/files/14232407/sysinfo-2024-02-11-12-43.json)
### Console logs
```Shell
Console logs added as files in "Additional information" section, let me know if I should instead just paste them here
```
### Additional information
[error_1-mixture_sd15_controlnet.txt](https://github.com/lllyasviel/stable-diffusion-webui-forge/files/14232448/error_1-mixture_sd15_controlnet.txt)
[error_2-multidiffusion_sd15_controlnet_tile_colorfix+sharp.txt](https://github.com/lllyasviel/stable-diffusion-webui-forge/files/14232449/error_2-multidiffusion_sd15_controlnet_tile_colorfix%2Bsharp.txt)
[error_3-sdxl.txt](https://github.com/lllyasviel/stable-diffusion-webui-forge/files/14232450/error_3-sdxl.txt)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in the img2img entry point with Multidiffusion Integrated and Controlnet Integrated enabled, then reproduce the listed SD1.5 and SDXL combinations. Compare the attached error_1, error_2, and error_3 console logs with the working combinations and inspect the relevant processing path. Done means the documented combinations complete image generation without errors and the Tile Batch Size range is verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100