Multi-LoRA Composite method
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 155
Description
### Feature Idea
Idea presented in a paper from earlier this year to combine multiple LoRAs in a more cohesive manner.
https://maszhongming.github.io/Multi-LoRA-Composition/
The paper presents three methods, referred to as Merge, Switch, and Composite. Merge is the current implementation in ComfyUI (merges LoRA weights then apply to model). Switch performs the same logic as Merge, but applies a single LoRA at each step. Composite calculates the cond and uncond for each LoRA individually at each denoise step, and averages each of these weights for a single denoise step -- **this is what this feature request is proposing**.
### Existing Solutions
Of the methods mentioned above:
- "Merge" is the method implemented in ComfyUI and [AUTOMATIC1111/stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui)
- "Switch" can be employed via [asagi4/comfyui-prompt-control](https://github.com/asagi4/comfyui-prompt-control)
- "Composite" is not available in any way to use in ComfyUI to my knowledge.
### Other
https://arxiv.org/abs/2402.16843
https://github.com/maszhongming/Multi-LoRA-Composition

Contributor guide
Research direction
No source file, test, or entry point is named. Read the linked paper and Multi-LoRA-Composition repository first, then trace ComfyUI's existing Merge implementation to determine where per-LoRA conditioning and denoising are handled. Done means Composite applies the proposed averaging behavior and is usable in ComfyUI.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100