Comfy-Org / Comfy-Org/ComfyUI

Node for Blending Sigmas

Open
#5,250 0 comments 0 reactions 0 assignees View on GitHub
Feature
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 10h
Merged PRs (30d)
153

Description

### Feature Idea

Different schedulers have different strengths and weaknesses. In my experiments with photography/realism prompts using Flux.1 Dev I've noticed that `beta` scheduler has very good prompt adherence but the output tends to be less diverse (certain prompts have a very similar look and feel across different seeds). By contrast `ddim_uniform` is a bit worse when it comes to adherence but the output looks more varied and creative and photographic images also look more "natural".

The question is: can we combine the best of both schedules? I think having a way to blend the schedule sigmas using linear interpolation can work for this purpose (to some extent).

### Existing Solutions

I've hacked together a modification of `BasicScheduler` node that implements the requested effect:
```python
class BlendedScheduler:
@classmethod
def INPUT_TYPES(s):
return {"required":
{"model": ("MODEL",),
"scheduler1": (comfy.samplers.SCHEDULER_NAMES, ),
"scheduler2": (comfy.samplers.SCHEDULER_NAMES, ),
"ratio": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01}),
"steps": ("INT", {"default": 20, "min": 1, "max": 10000}),
"denoise": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}),
}
}
RETURN_TYPES = ("SIGMAS",)
CATEGORY = "sampling/custom_sampling/schedulers"

FUNCTION = "get_sigmas"

def get_sigmas(self, model, scheduler1, scheduler2, ratio, steps, denoise):
total_steps = steps
if denoise < 1.0:
if denoise <= 0.0:
return (torch.FloatTensor([]),)
total_steps = int(steps/denoise)

sigmas1 = comfy.samplers.calculate_sigmas(model.get_model_object("model_sampling"), scheduler1, total_steps).cpu()
sigmas1 = sigmas1[-(steps + 1):]
sigmas2 = comfy.samplers.calculate_sigmas(model.get_model_object("model_sampling"), scheduler2, total_steps).cpu()
sigmas2 = sigmas2[-(steps + 1):]

sigmas = torch.lerp(sigmas1, sigmas2, ratio)
return (sigmas, )
```

It probably makes more sense to have something that takes the sigmas as input instead?

### Other

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.