lllyasviel / lllyasviel/stable-diffusion-webui-forge
[Feature Request]: Batch Unet then Batch VAE
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
### Is there an existing issue for this?
- [X] I have searched the existing issues and checked the recent builds/commits
### What would your feature do ?
Currently, when doing high `Batch count` *(**not** `Batch size`)*, it would unload the model and load the encoder, then unload the encoder and load the model again, between every single generation. This reloading takes `~1.5` second in total on my system, slowing down the overall workflow.
Is it possible to add a Toggle, making it only switch to the VAE after all latents are finished processing?
Maybe store the intermediate latents in system RAM in the meantime?
### Proposed workflow
1. Enable some new Toggle
2. Generate with high `Batch count`
3. See all batches of latents being processed first
4. Then see all latents being converted to images
5. Observe only 1 occurance of loading model
### Additional information
Example of Current Console between each Generation:
```c
To load target model AutoencoderKL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6983.69580078125
[Memory Management] Model Memory (MB) = 159.55708122253418
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 5800.138719558716
Moving model(s) has taken 0.10 seconds
To load target model SDXL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6983.6875
[Memory Management] Model Memory (MB) = 4897.086494445801
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 1062.6010055541992
Moving model(s) has taken 1.24 seconds
```
**Edit:** Apparently this only happens when using SDXL checkpoints. Is it because my VRAM is barely enough?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue does not name files or tests. Start by tracing the high Batch count workflow and the SDXL model/VAE loading shown in the console, then determine where a toggle could defer VAE loading. Done means batches process their latents first, images are converted afterward, and model loading occurs only once.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100