lllyasviel / lllyasviel/stable-diffusion-webui-forge

[Bug]: Error with ControlNet preprocessor "tile_colorfix"

Open
#585 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
13k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

### Checklist

- [X] The issue exists after disabling all extensions
- [X] The issue exists on a clean installation of webui
- [X] The issue is caused by an extension, but I believe it is caused by a bug in the webui
- [X] The issue exists in the current version of the webui
- [X] The issue has not been reported before recently
- [ ] The issue has been reported before but has not been fixed yet

### What happened?

in built in ControlNet extension, select preprocdessor "tile_colorfix" or "tile_colorfix+sharp" can reproduce the error.

### Steps to reproduce the problem

in built in ControlNet extension, select preprocdessor "tile_colorfix" or "tile_colorfix+sharp" can reproduce the error.
企业微信20240320-192736@2x

### What should have happened?

error in built in ControlNet extension's preprocessor

### What browsers do you use to access the UI ?

Google Chrome

### Sysinfo

version: [f0.0.16v1.8.0rc-1.7.0](https://github.com/lllyasviel/stable-diffusion-webui-forge/commit/da9663c6023e48a91d929d32bf5b64f75e7d902d)  •  python: 3.11.7  •  torch: 2.2.0  •  xformers: N/A  •  gradio: 3.41.2  •  checkpoint: [354b8c571d](https://google.com/search?q=354b8c571d3abe963e1520d3b4e0647be519841f4376fc5d16f7f5e7859f7d49)

### Console logs

```Shell
2024-03-20 19:25:27,228 - ControlNet - INFO - ControlNet Input Mode: InputMode.SIMPLE
2024-03-20 19:25:27,230 - ControlNet - INFO - Using preprocessor: tile_colorfix
2024-03-20 19:25:27,230 - ControlNet - INFO - preprocessor resolution = 0.5
2024-03-20 19:25:27,297 - ControlNet - INFO - Current ControlNet ControlNetPatcher: /mnt/data/chenzonghao/Projects1/stable-diffusion-webui-forge/models/ControlNet/sd15/control_v11f1e_sd15_tile.pth
Warning: field include_init_images in API payload not found in .
Warning: field infotext in API payload not found in .
2024-03-20 19:25:27,926 - ControlNet - INFO - ControlNet Method tile_colorfix patched.
To load target model BaseModel
To load target model ControlNet
Begin to load 2 models
Reuse 1 loaded models
Reuse 1 loaded models
[Memory Management] Current Free GPU Memory (MB) = 15322.3984375
[Memory Management] Model Memory (MB) = 0.0
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 14298.3984375
[Memory Management] Current Free GPU Memory (MB) = 15322.3984375
[Memory Management] Model Memory (MB) = 0.0
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 14298.3984375
Moving model(s) has taken 0.02 seconds
0%| | 0/23 [00:00
samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/docker/chenzonghao/miniconda3/envs/forge/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/mnt/data/chenzonghao/Projects1/stable-diffusion-webui-forge/repositories/k-diffusion/k_diffusion/sampling.py", line 594, in sample_dpmpp_2m
denoised = model(x, sigmas[i] * s_in, **extra_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/docker/chenzonghao/miniconda3/envs/forge/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/docker/chenzonghao/miniconda3/envs/forge/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/data/chenzonghao/Projects1/stable-diffusion-webui-forge/modules/sd_samplers_cfg_denoiser.py", line 182, in forward
*** ...,
*** [0, 0, 0],
*** [0, 0, 0],
*** [0, 0, 0]],
***
*** [[0, 0, 0],
*** [0, 0, 0],
*** [0, 0, 0],
*** ...,
*** [0, 0, 0],
*** [0, 0, 0],
*** [0, 0, 0]],
***
*** [[0, 0, 0],
*** [0, 0, 0],
*** [0, 0, 0],
*** ...,
*** [0, 0, 0],
*** [0, 0, 0],
*** [0, 0, 0]]], dtype=uint8)}, resize_mode='Just Resize', processor_res=0.5, threshold_a=0.5, threshold_b=0.5, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', save_detected_map=True), ControlNetUnit(input_mode=, use_preview_as_input=False, batch_image_dir='', batch_mask_dir='', batch_input_gallery=[], batch_mask_gallery=[], generated_image=None, mask_image=None, hr_option='Both', enabled=False, module='None', model='None', weight=1, image=None, resize_mode='Crop and Resize', processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', save_detected_map=True), False, 7, 1, 'Constant', 0, 'Constant', 0, 1, 'enable', 'MEAN', 'AD', 1, False, 1.01, 1.02, 0.99, 0.95, False, 0.5, 2, False, 256, 2, 0, False, False, 3, 2, 0, 0.35, True, 'bicubic', 'bicubic', False, 0, 'anisotropic', 0, 'reinhard', 100, 0, 'subtract', 0, 0, 'gaussian', 'add', 0, 100, 127, 0, 'hard_clamp', 5, 0, 'None', 'None', False, 'MultiDiffusion', 768, 768, 64, 4, False, False, False, '* `CFG Scale` should be 2 or lower.', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', '

Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8

', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, 'start', '', '

Will upscale the image by the selected scale factor; use width and height sliders to set tile size

', 64, 0, 2, 1, '', [], 0, '', [], 0, '', [], True, False, False, False, False, False, False, 0, False) {}
Traceback (most recent call last):
File "/mnt/data/chenzonghao/Projects1/stable-diffusion-webui-forge/modules/call_queue.py", line 57, in f
res = list(func(*args, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable
```

### Additional information

_No response_

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the failure by selecting the built-in ControlNet preprocessor tile_colorfix or tile_colorfix+sharp, then start at modules/sd_samplers_cfg_denoiser.py:182 and follow the call path through modules/processing.py and modules/sd_samplers_kdiffusion.py. Use the supplied traceback and console output to identify why the operation returns None; done means the preprocessor completes without the TypeError and generates the image.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.