Way of releasing cached outputs/GC to free up resources within the same graph execution
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 155
Description
Some more complex workflows may involve loading different diffusion models in a sequence so they can be combined in a HR-fix manner. However as far as I know any previous models loaded in the graph will remain loaded for the rest of the execution. This makes sense for caching small outputs like image filepaths but can lead to OOM situations for VRAM.
Could there be a way of telling the graph to discard a certain output and immediately GC the graphics memory after it's finished if the user requests it? It could be an opt-in feature. Litegraph has node properties that you can set, which aren't displayed as widgets on the node but can be manipulated with the right-click menu. Could add a `releaseCachedOutput` toggle, such that when the graph execution is calculated, the last time the output is used by any node it will be released. This will mean you always have to reload those outputs when re-running the graph but it enables running more complex workflows without being limited by total VRAM allocated over the whole graph.
Contributor guide
Assessment
This issue has not been assessed yet.