kohya-ss / kohya-ss/sd-scripts

82713e9 update using more vram in lora training?

Open
#197 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.2k
Forks
1.2k
Avg merge
11m
Merged PRs (30d)
2

Description

I can train lora model in [b32abdd](https://github.com/kohya-ss/sd-scripts/commit/b32abdd327f314fabc76fbbb627218090afb7b69) version using rtx3050 4g laptop with `--xformers --shuffle_caption --use_8bit_adam --network_train_unet_only --mixed_precision="fp16"`

but when I update to [82713e9](https://github.com/kohya-ss/sd-scripts/commit/82713e9aa62d9a28b17b3920b2b9bccd74051f9b) version(which is lastest) I just out of memory before _`Replace CrossAttention.forward to use xformers`_ hint

I can still training on [b32abdd](https://github.com/kohya-ss/sd-scripts/commit/b32abdd327f314fabc76fbbb627218090afb7b69) version, but I want to know which part of changes cause that?

Contributor guide

No contributing guide indexed for this repository

Research direction

Compare commits b32abdd and 82713e9, then reproduce the reported LoRA training command on the RTX 3050 4GB setup and observe where memory usage changes before the “Replace CrossAttention.forward to use xformers” hint. Done means identifying the change that causes the out-of-memory failure and documenting the relevant evidence.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.