kohya-ss / kohya-ss/sd-scripts
How to keep model weights loaded in GPU?
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 1.2k
- Avg merge
- 11m
- Merged PRs (30d)
- 2
Description
Hello, I'm using gen_img_diffusers.py and it's pretty cool to generate great images with that script, But when i want to generate a few images it re-loads the models (tokenizer, unet, etc) and lates too much between image generations.
Isn't there a way to keep all those information in GPU just like a Pipeline?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading gen_img_diffusers.py and trace how the tokenizer, UNet, and other models are initialized for each image generation. Check whether the script already exposes a reusable pipeline or generation loop. Done means multiple images can be generated without reloading the models into GPU memory between generations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100