deforum / deforum/sd-webui-deforum

[Feature Request]: color coherence mode — backpass with interpolation

Open
#401 4 comments 0 reactions 0 assignees View on GitHub
enhancement output video input
Dominant language
Python
Stars
2.9k
Forks
407
PR merge metrics
No merged PRs in 30d

Description

### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits

### What would your feature do ?

Color coherence is a huge issue in vid2vid mode of Deforum. If you switch it to None, the colors will basically flicker each frame as you see here https://youtu.be/U9lRAyHwBiQ and color coherence on kills the variety of the palette. Color coherence each N steps don't help much it will just delay the flicker.

What I suggest: render a sequence, then grab the histograms of the finished frame and of the previous one at the selected distance. Either interpolate the histograms somehow, or render an interpolation of the start and the end frames with film or rife and then transfer the histograms using cv match_histograms

### Proposed workflow

1. Go to coherence section
2. Press 'backpass', enter the reasonable window size
3. See the animation with **no color flicker** and *more creative colors responding to the prompts*

### Additional information

_No response_

### Are you going to help adding it? If not, tell why.

Sure

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.