kohya-ss / kohya-ss/sd-scripts
Support individual lora module level selection and settings for more granular training
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 1.2k
- Avg merge
- 11m
- Merged PRs (30d)
- 2
Description
Applying a global values for the training of text encoders and unet can lead to bad training results in some cases. By allowing different learning rates, dimensions, and alphas, for individual lora modules as well as choosing which modules to train on each block, not just as a group, will allow for better training results to be achieved. Currently, I edit the lora.py file in order to achieve this but having official support for this would be great.
For example, training the text encoder self attention modules can lead to overfitting quickly. There's currently no option to disable those specific modules or adjust the learning rate for them. In an older releases until about half a year ago, there was a bug that prevented the training of these modules, which inadvertently result in loras with noticeably better fidelity. Since this bug was fixed, it was impossible to reproduce such results because these text encoder self attention modules are always trained, and there's no option to configure them.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing lora.py, especially how current global text encoder and UNet training values are applied. Map the existing module groups and determine how per-module selection, learning rates, dimensions, and alphas would be configured. Done means users can independently enable or disable modules and set those values without editing lora.py directly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100