huggingface / huggingface/diffusers

A new pipeline for inpainting(object removal) is needed!!

Open
#4,836 8 comments 1 reaction 0 assignees View on GitHub
community-examples New pipeline/model
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

**Is your feature request related to a problem? Please describe.**
Thanks for your great work!!
Right now I'm using StableDiffusionInpaintPipeline with fixed prompt(such as "the empty background") to do object removal task, but one thing i've found is that **when the inpaint mask doesn't cover the shadow or hairline, the model will randomly generate objects else in the mask rather than generating pure background,** which is frustrating. I wonder if the inpaint pipeline is trained for removing object ? If not, when will the inpaint model trained for object removal be released(works like LDM inpainting or LaMa)? As this feature can be really useful in practice!

**Describe the solution you'd like**
Plz release the inpaint pipeline specialized for object removal!

**Describe alternatives you've considered**
Give some advice about getting rid of unwanted object generation

unwanted object generation:
![A57CD3D6-DA0C-4FA3-9A88-373C7E21AC9A](https://github.com/huggingface/diffusers/assets/52605055/7e5a4537-cf4a-4dbd-b4b6-ae9995ff64b7)
![F84FF064-F0D9-43DA-83A4-BF4FACB5E910](https://github.com/huggingface/diffusers/assets/52605055/66d54ad9-77a1-43ec-8c61-db7915292198)
![7CB3DC41-76FF-40CF-B32A-1D3D4A309D50](https://github.com/huggingface/diffusers/assets/52605055/669ac226-78bd-4c61-bfcc-b8271df64d53)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.