lllyasviel / lllyasviel/stable-diffusion-webui-forge
Forge BE question - switching between models
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
Hi all. I'm not sure this is an issue, but the thing I've noticed is that loading of models (switching between them) is what's taking most of the time for me during image generation. I'm using the same model (checkpoint), same ClipSkip. But this model are loaded all the time, multiple times during generation, each generation.
Is there a way to avoid that ? I mean I don't change models, why system is loading them each time from the scratch ? wtf ?
Example of 1 banch of model swtiching (have them during 1 image generation)
```
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6963.8125
[Memory Management] Model Memory (MB) = 159.55708122253418
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 5780.255418777466
Moving model(s) has taken 0.66 seconds
0: 448x640 1 eyes, 25.6ms
Speed: 44.1ms preprocess, 25.6ms inference, 48.9ms postprocess per image at shape (1, 3, 448, 640)
To load target model SDXLClipModel
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6781.39453125
[Memory Management] Model Memory (MB) = 2144.3546981811523
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 3613.0398330688477
Moving model(s) has taken 13.33 seconds
To load target model SDXL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6886.24365234375
[Memory Management] Model Memory (MB) = 4897.086494445801
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 965.1571578979492
Moving model(s) has taken 29.50 seconds
100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:06<00:00, 1.40it/s]
To load target model AutoencoderKL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6937.57470703125
[Memory Management] Model Memory (MB) = 159.55708122253418
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 5754.017625808716
Moving model(s) has taken 0.74 seconds
To load target model SDXL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6937.80078125
[Memory Management] Model Memory (MB) = 4897.086494445801
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 1016.7142868041992
Moving model(s) has taken 2.39 seconds
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:13<00:00, 1.51it/s]
To load target model AutoencoderKL█████████████████████████████████████████████████████| 40/40 [01:24<00:00, 1.46it/s]
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6937.56640625
[Memory Management] Model Memory (MB) = 159.55708122253418
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 5754.009325027466
Moving model(s) has taken 0.37 seconds
0: 448x640 2 eyess, 26.8ms
Speed: 37.0ms preprocess, 26.8ms inference, 61.4ms postprocess per image at shape (1, 3, 448, 640)
To load target model SDXLClipModel
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6774.3173828125
[Memory Management] Model Memory (MB) = 2144.3546981811523
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 3605.9626846313477
Moving model(s) has taken 14.44 seconds
To load target model SDXL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6907.81103515625
[Memory Management] Model Memory (MB) = 4897.086494445801
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 986.7245407104492
Moving model(s) has taken 16.53 seconds
100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:06<00:00, 1.36it/s]
To load target model AutoencoderKL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6933.12255859375
[Memory Management] Model Memory (MB) = 159.55708122253418
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 5749.565477371216
Moving model(s) has taken 0.58 seconds
To load target model SDXLClipModel
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6770.10791015625
[Memory Management] Model Memory (MB) = 2144.3546981811523
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 3601.7532119750977
Moving model(s) has taken 6.04 seconds
To load target model SDXL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6903.6015625
[Memory Management] Model Memory (MB) = 4897.086494445801
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 982.5150680541992
Moving model(s) has taken 25.67 seconds
100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:06<00:00, 1.40it/s]
To load target model AutoencoderKL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6928.9130859375
[Memory Management] Model Memory (MB) = 159.55708122253418
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 5745.356004714966
Moving model(s) has taken 0.80 seconds
Total progress: 100%|██████████████████████████████████████████████████████████████████| 40/40 [02:49<00:00, 4.25s/it]
Cleanup minimal inference memory.██████████████████████████████████████████████████████| 40/40 [02:49<00:00, 1.46it/s]
tiled upscale: 100%|███████████████████████████████████████████████████████████████████| 35/35 [00:08<00:00, 3.98it/s]
To load target model SDXLClipModel
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6766.72705078125
[Memory Management] Model Memory (MB) = 2144.3546981811523
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 3598.3723526000977
Moving model(s) has taken 10.50 seconds
To load target model SDXL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6927.48779296875
[Memory Management] Model Memory (MB) = 4897.086494445801
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 1006.4012985229492
Moving model(s) has taken 29.45 seconds
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:13<00:00, 1.49it/s]
To load target model AutoencoderKL████████████████████
```
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The report names no files, tests, or entry points; begin by reproducing the repeated SDXLClipModel, SDXL, and AutoencoderKL loads from the provided log. Trace the model-loading path and verify that repeated generation with the same checkpoint no longer reloads unchanged models.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100