kohya-ss / kohya-ss/sd-scripts

How to keep model weights loaded in GPU?

Open
#486 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.2k
Forks
1.2k
Avg merge
11m
Merged PRs (30d)
2

Description

Hello, I'm using gen_img_diffusers.py and it's pretty cool to generate great images with that script, But when i want to generate a few images it re-loads the models (tokenizer, unet, etc) and lates too much between image generations.

Isn't there a way to keep all those information in GPU just like a Pipeline?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading gen_img_diffusers.py and trace how the tokenizer, UNet, and other models are initialized for each image generation. Check whether the script already exposes a reusable pipeline or generation loop. Done means multiple images can be generated without reloading the models into GPU memory between generations.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.