Comfy-Org / Comfy-Org/ComfyUI

Different Conditioning for each Transformer block

Open
#9,005 0 comments 0 reactions 0 assignees View on GitHub
bug-cop:non-blocking Core Feature
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Feature Idea

From my research, some transformer blocks is very sensitive and fragile to emphasized conditioning.
If we can feed different conditioning to such transformer blocks, we can use much stronger emphasis.
However, to use above technique, current Comfyui implementation has two problems:
1. I have to encode model conds manually. I don't want to do it as it would be compatible issue.
2. Comfyui backend may split batches which we cannot deal with.

Please make your implementation easier to deal with above problems.

### Existing Solutions

_No response_

### Other

Examples:
Feeding same conditioning to every transformer blocks
Image

Feeding emphasized conditioning only to /model.diffusion_model.output_blocks\.0\.1\.transformer_blocks\.[0-9]/ .
Image

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.