lllyasviel / lllyasviel/stable-diffusion-webui-forge
"For inpainting, overlay the original image over the areas that were untouched" creates artifacts on high resolution images
- Dominant language
- Python
- Stars
- 13k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
There are 3 main steps of inpainting:
1. Cutting the chosen bbox and upscaling it to chosen resolution.
2. Inpaint.
3. Downscaling the result of inpaint and blending inpaint result with original image.
The last step creates artifacts, which are very visible, if the resolution of the image is quite high. It's probably a flawed downscale method.
The artifacts are visible lines around inpainted area which also change the color a bit.
The same problem happens with Adetailer, for example, it creates artifacts around the bbox that Adetailer "chose" for inpaint.
I'll post a simple example, but these artifacts are more or less visible depending on the inpaint area and color/brightness.
This is the initial image:

I used inpaint near it's left hand:

Here's the result, it's not really noticeable:

But if I take the inpaint area and increase contrast, you can see a large circle artifact around inpainted area:

And if I turn off the option "For inpainting, overlay the original image over the areas that were untouched", then the inpainted image gets saved separately and doesn't get downscaled, and there are no artifacts:

I tried changing many different settings, and none of them helped. I hope this gets fixed, because I inpaint high resolution images a lot.
I should also mention, that if I turn on "For inpainting, save the masked composite" option and look at the result, then it also has artifacts, but inside the inpainted area:

These artifacts probably disappear during "blending" phase (because of that I decided that it's the flawed downscale method and not the blending phase), but the other ones (mentioned above) stay on the result image.
Also, if the cut bbox doesn't get upscaled and downscaled (if it's size is equal to width/height you set in UI) then this bug doesn't occur, there are no artifacts.
This problem occurs in Forge, Reforge and Forge Classic. I'll post this issue in all UIs that are currently in development.
Thank you for attention!
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the issue with a high-resolution image, an upscaled and downscaled inpaint bounding box, and the “For inpainting, overlay the original image over the areas that were untouched” option enabled. Trace the resize and blending path described in the report, comparing it with the option disabled; done means the visible boundary artifacts no longer appear after inpainting.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100