Comfy-Org / Comfy-Org/ComfyUI

"Pinned_memory" feature is amazing!!!

Open
#10,555 43 comments 5 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

Hi ComfyUI team! This is not an issue but a positive feedback about recent changes in ComfyUI code. In particular about `--fast pinned_memory` but also about `--async-offload`.

My system is:
Windows 11
CPU: AMD Ryzen 7 5700X 8 core
RAM: 64 GB
GPU: RTX 5060 Ti 16GB on PCIe Gen3 @ x8 (Gen3 it's maximum on my AM4 motherboard, @ x8 is maximum on RTX 5060 series)
Resizable BAR: enabled in UEFI and in nvidia Profile Inspector

Let's look at this Task Manager picture:

Image

In the past when GPU Memory was getting to any tiny amount higher than the actual VRAM of my card (15.x-16 GB) I would not get an OOM right away (thanks to default Nvidia driver behavior in Windows to fall back on system RAM) but any processing from that point onward would be slowing down to a frozen jelly speed, taking forever to finish a VAE decoding for example. And that in spite of the fact that Task Manager would show 100% GPU utilization. But you could tell that it not exactly so looking at the temperature, it would show something in 44-47 C range, meaning that GPU was running at idle temperatures. Now look at the screenshot again an observe the temperature: 70 C. Amazing, I don't know what kind of sorcery is this but is very welcome! And I get the subjective feeling that my GPU is more effectively run even at times and steps in the workflow when the actual hardware GPU VRAM is not exceeded.

I am planning to propose a similar enhancement to Llama.cpp project, if that would be possible, since Llama.cpp has exactly the same behavior like ComfyUI had in the past. A loaded LLM model with a 16.1 GB VRAM utilization would take forever to process any prompt forcing me to use only less demanding models that would occupy only 8 - 12 GB VRAM.

Also I would like to thank about the changes in `--async-offload`. The performance and behavior on my system is finally better with it enabled. In the past I had issues with slow speed and / or noise output in various workflows so I kind of excluded `--async-offload` permanently from my options to enhance performance on my system. But now the situation seems much better. I didn't make any measurements but with my usual workflows it seems to at least not cause any downside. Since I enabled both options simultaneously it's hard to tell which impact has each one separately, but I can tell that the first run of a Wan 2.2 I2V workflow (using unquantized fp16 safetensors) that before the latest ComfyUI updates was taking about 19 minutes, now it takes a little over 16 minutes. Probably the difference is: faster loading from disk, better RAM and VRAM utilization, better GPU time utilization. At least, lacking objective benchmarks, is what is feeling like just looking at Task Manager.

Again, many thanks to ComfyUI team!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.