huggingface / huggingface/diffusers

Differential Diffusion with ControlNet pipeline

Open
#9,186 4 comments 0 reactions 1 assignee Claimed by @asomoza View on GitHub
stale
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

**Is your feature request related to a problem? Please describe.**
Many users have asked me whether diff diff goes well with ControlNet :
https://github.com/exx8/differential-diffusion/issues/29,
https://github.com/exx8/differential-diffusion/issues/19
It does!
I have conducted an experiment in the past, but it was with an old diffusion model (SD2).
I believe that ControlNet and diff diff can give yet to be seen control over image editing:
I tend to imagine it as diff diff control how far the inference process can go from the origin in a region, while ControlNet control to what direction.
So, it is expected to be quite powerful and useful.
The implementation is surprising straightforward, diff diff and ControlNet don't collide.
**Describe the solution you'd like.**
A user will be able to use both ControlNet and diff diff for the same inference process.

**Describe alternatives you've considered.**
Using diff diff alone or ControlNet alone is the only alternative I can think of. This won't get the same amount of control as the combination of both.

**Additional context.**
I can release my code if it will help, but it contains a little bug, and it is for an old version.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.