lllyasviel / lllyasviel/stable-diffusion-webui-forge

"For inpainting, overlay the original image over the areas that were untouched" creates artifacts on high resolution images

Open
#2,889 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
13k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

There are 3 main steps of inpainting:
1. Cutting the chosen bbox and upscaling it to chosen resolution.
2. Inpaint.
3. Downscaling the result of inpaint and blending inpaint result with original image.

The last step creates artifacts, which are very visible, if the resolution of the image is quite high. It's probably a flawed downscale method.
The artifacts are visible lines around inpainted area which also change the color a bit.
The same problem happens with Adetailer, for example, it creates artifacts around the bbox that Adetailer "chose" for inpaint.
I'll post a simple example, but these artifacts are more or less visible depending on the inpaint area and color/brightness.

This is the initial image:

![Image](https://github.com/user-attachments/assets/a515c018-1373-406c-9691-d59bd1f31fa8)

I used inpaint near it's left hand:

![Image](https://github.com/user-attachments/assets/faf04518-7dfa-4b7a-b2fb-7aa7fb185a2b)

Here's the result, it's not really noticeable:

![Image](https://github.com/user-attachments/assets/1d1a1475-2230-4461-bf31-50b4329abd4c)

But if I take the inpaint area and increase contrast, you can see a large circle artifact around inpainted area:

![Image](https://github.com/user-attachments/assets/b7c85d73-21f6-4d7a-b556-cf5ed22d9166)

And if I turn off the option "For inpainting, overlay the original image over the areas that were untouched", then the inpainted image gets saved separately and doesn't get downscaled, and there are no artifacts:

![Image](https://github.com/user-attachments/assets/98e07e4f-b195-4f74-886f-5e631722cb56)

I tried changing many different settings, and none of them helped. I hope this gets fixed, because I inpaint high resolution images a lot.

I should also mention, that if I turn on "For inpainting, save the masked composite" option and look at the result, then it also has artifacts, but inside the inpainted area:

![Image](https://github.com/user-attachments/assets/9cf6619d-d872-46ae-8844-90e9c9bbc3bf)

These artifacts probably disappear during "blending" phase (because of that I decided that it's the flawed downscale method and not the blending phase), but the other ones (mentioned above) stay on the result image.
Also, if the cut bbox doesn't get upscaled and downscaled (if it's size is equal to width/height you set in UI) then this bug doesn't occur, there are no artifacts.

This problem occurs in Forge, Reforge and Forge Classic. I'll post this issue in all UIs that are currently in development.
Thank you for attention!

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the issue with a high-resolution image, an upscaled and downscaled inpaint bounding box, and the “For inpainting, overlay the original image over the areas that were untouched” option enabled. Trace the resize and blending path described in the report, comparing it with the option disabled; done means the visible boundary artifacts no longer appear after inpainting.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.