kohya-ss / kohya-ss/sd-scripts
[Feature Request] Implement ZipLoRA Merge Algorithm
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 1.2k
- Avg merge
- 11m
- Merged PRs (30d)
- 2
Description
ZipLoRA, a novel method for merging low-rank adaptations (LoRAs) to achieve high-fidelity, concept-driven personalization in generative models, has shown promising results in maintaining style and subject integrity. I propose integrating this algorithm into sd-scripts to enhance its capabilities in generating personalized stylizations.
ZipLoRA leverages a "zipper-like" approach to blend two LoRAs, minimizing similar-direction sums and preserving original content and style properties. This is based on observations from recent generative model developments and the sparsity of LoRA weight matrices.
**Implementation References:**
- **Diffusers Implementation**: https://github.com/mkshing/ziplora-pytorch
- **Paper Page**: https://ziplora.github.io/
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the linked Diffusers implementation at github.com/mkshing/ziplora-pytorch and the ZipLoRA paper page to understand the algorithm and its expected inputs. Then locate the LoRA merging entry point in sd-scripts. Done means the algorithm is integrated into sd-scripts and can merge two LoRAs while preserving the stated style and subject properties.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100