lllyasviel / lllyasviel/stable-diffusion-webui-forge

[Bug]: Fooocus inpaint does not work

Open
#256 7 comments 2 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
13k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

### Checklist

- [ ] The issue exists after disabling all extensions
- [X] The issue exists on a clean installation of webui
- [ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
- [X] The issue exists in the current version of the webui
- [X] The issue has not been reported before recently
- [X] The issue has been reported before but has not been fixed yet

### What happened?

Fooocus inpaint, The resulting results change very little

![PixPin_2024-02-15_01-14-59](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/139383150/8bb0d4a5-3c3a-42c7-963f-3e116e906ef1)

### Steps to reproduce the problem

![PixPin_2024-02-15_01-16-49](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/139383150/42d5f90c-1432-4def-ba92-8841713314b8)
![PixPin_2024-02-15_01-17-21](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/139383150/52ba5481-b4ad-43c3-8176-f99172d6a133)

long_hair,
Steps: 7, Sampler: DPM++ SDE Karras, CFG scale: 1.5, Seed: 2368543817, Size: 712x1024, Model hash: 7eb187c0d0, Model: dreamshaperXL_v2TurboDpmppSDE, Denoising strength: 1, Clip skip: 2, ControlNet 0: "Module: inpaint_only, Model: inpaint_v26.fooocus [be360607], Weight: 1, Resize Mode: Crop and Resize, Processor Res: 0.5, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0, Guidance End: 0.5, Pixel Perfect: False, Control Mode: ControlNet is more important, Hr Option: Both", Mask blur: 4, Masked content: fill, Version: f0.0.12-latest-155-gd81e353d

Time taken: 11.2 sec.

### What should have happened?

![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/139383150/af78753b-b89a-46d9-b332-8291373f4cda)

### What browsers do you use to access the UI ?

Microsoft Edge

### Sysinfo

[sysinfo-2024-02-14-17-31.json](https://github.com/lllyasviel/stable-diffusion-webui-forge/files/14284320/sysinfo-2024-02-14-17-31.json)

### Console logs

```Shell
/root/stable-diffusion-webui-forge
Python 3.10.8 (main, Nov 24 2022, 14:13:03) [GCC 11.2.0]
Version: f0.0.12-latest-155-gd81e353d
Commit hash: d81e353d8928147bbd973068d0efbb2802affe0f
Launching Web UI with arguments: --port=6006 --theme=dark --no-half --xformers
Total VRAM 20188 MB, total RAM 257808 MB
xformers version: 0.0.23.post1
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3080 : native
VAE dtype: torch.bfloat16
/root/miniconda3/lib/python3.10/site-packages/pytorch_lightning/utilities/distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead.
rank_zero_deprecation(
Using xformers cross attention
ControlNet preprocessor location: /root/stable-diffusion-webui-forge/models/ControlNetPreprocessor
sd-webui-prompt-all-in-one background API service started successfully.
Loading weights [7eb187c0d0] from /root/stable-diffusion-webui-forge/models/Stable-diffusion/dreamshaperXL_v2TurboDpmppSDE.safetensors
2024-02-15 01:27:54,992 - ControlNet - INFO - ControlNet UI callback registered.
Running on local URL: http://127.0.0.1:6006
model_type EPS
UNet ADM Dimension 2816
Using xformers attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using xformers attention in VAE

To create a public link, set `share=True` in `launch()`.
Startup time: 19.4s (prepare environment: 3.1s, import torch: 8.6s, other imports: 0.6s, load scripts: 3.0s, create ui: 1.2s, gradio launch: 2.4s, app_started_callback: 0.4s).
extra {'cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_g.transformer.text_model.embeddings.position_ids', 'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_l.text_projection'}
left over keys: dict_keys(['conditioner.embedders.0.logit_scale', 'conditioner.embedders.0.text_projection'])
To load target model SDXLClipModel
Begin to load 1 model
Moving model(s) has taken 0.34 seconds
Model loaded in 6.4s (load weights from disk: 1.5s, forge load real models: 3.6s, load VAE: 0.2s, calculate empty prompt: 1.0s).
To load target model AutoencoderKL
Begin to load 1 model
To load target model SDXL
Begin to load 1 model
Moving model(s) has taken 0.91 seconds
0%| | 0/7 [00:00

Contributor guide

No contributing guide indexed for this repository

Research direction

No source file or test is named. Reproduce the issue with the listed SDXL model and settings, then trace the ControlNet `inpaint_only` path and `FooocusInpaintPatcher` shown in the logs. Done means Fooocus inpainting produces a result that meaningfully follows the mask and requested prompt.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.