Comfy-Org / Comfy-Org/ComfyUI

Multi-LoRA Composite method

Open
#5,691 0 comments 6 reactions 0 assignees View on GitHub
Feature
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 6h
Merged PRs (30d)
155

Description

### Feature Idea

Idea presented in a paper from earlier this year to combine multiple LoRAs in a more cohesive manner.
https://maszhongming.github.io/Multi-LoRA-Composition/

The paper presents three methods, referred to as Merge, Switch, and Composite. Merge is the current implementation in ComfyUI (merges LoRA weights then apply to model). Switch performs the same logic as Merge, but applies a single LoRA at each step. Composite calculates the cond and uncond for each LoRA individually at each denoise step, and averages each of these weights for a single denoise step -- **this is what this feature request is proposing**.

### Existing Solutions

Of the methods mentioned above:
- "Merge" is the method implemented in ComfyUI and [AUTOMATIC1111/stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui)
- "Switch" can be employed via [asagi4/comfyui-prompt-control](https://github.com/asagi4/comfyui-prompt-control)
- "Composite" is not available in any way to use in ComfyUI to my knowledge.

### Other
https://arxiv.org/abs/2402.16843
https://github.com/maszhongming/Multi-LoRA-Composition

![example](https://github.com/user-attachments/assets/e0493a4c-5be8-4057-99e7-f9374fadc1a9)

Contributor guide

Open the contributing guide

Research direction

No source file, test, or entry point is named. Read the linked paper and Multi-LoRA-Composition repository first, then trace ComfyUI's existing Merge implementation to determine where per-LoRA conditioning and denoising are handled. Done means Composite applies the proposed averaging behavior and is usable in ComfyUI.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.