Comfy-Org / Comfy-Org/Nvidia_RTX_Nodes_ComfyUI
RTX VideoSuperResolution node leaks GPU memory per-frame — deadlocks/hangs on long video runs
- Dominant language
- Python
- Stars
- 620
- Forks
- 38
- PR merge metrics
- No merged PRs in 30d
Description
## What's happening
The `RTXVideoSuperResolution` node works fine for a single image or a short batch, but when you feed it a longer video (or a batch of ~60+ images) it slowly eats all available GPU memory and then hangs/deadlocks. This is the same underlying problem reported in #16 ("slowly eat up all my system memory… swapping death loop"), just seen from the GPU side.
## Root cause
In `__init__.py`, the `execute()` method loops over every frame and calls `sr.run(input_frame)`:
```python
for j in range(batch_cuda.shape[0]):
input_frame = batch_cuda[j]
dlpack_out = sr.run(input_frame).image
out_tensor[i + j: i + j + 1] = torch.from_dlpack(dlpack_out).movedim(0, -1).unsqueeze(0)
```
`sr.run()` returns a dlpack tensor. `torch.from_dlpack(dlpack_out)` creates a **new** GPU tensor that *shares* the dlpack's underlying storage, and that dlpack reference is reassigned on the next iteration but the previous one is never explicitly freed. Because the converted PyTorch tensor aliases the same memory, the allocator can't reclaim the storage either. Over many frames the per-frame allocations accumulate in the CUDA caching allocator and never get returned, so VRAM climbs monotonically until the node OOMs or deadlocks waiting on a CUDA call.
The `.movedim().unsqueeze()` ops also produce intermediate views rather than owning their own storage, so the original allocation's lifetime stays pinned to the loop.
## Reproduction
1. Open ComfyUI with the RTX nodes installed on an NVIDIA GPU.
2. Load a video longer than ~60 frames (or generate a batch of 60+ images).
3. Wire it into an `RTX Video Super Resolution` node at any scale > 1x.
4. Run the graph and watch VRAM in `nvidia-smi` — it climbs steadily and never drops until the process is killed. On cards with less headroom the node simply hangs partway through.
Observed on Ubuntu 24.04, RTX 4080, 48 GB RAM (per #16) and reproduced locally on other configs.
## Expected behavior
VRAM usage should stay roughly flat regardless of frame count — each frame's intermediates should be freed before the next frame is processed.
## Proposed fix
Three changes to the per-frame loop:
1. **`.clone()` the converted tensor** so the output owns its own storage and the dlpack-backed tensor can be released immediately.
2. **`del` the dlpack/result/out_frame references** explicitly so CPython drops them on this iteration instead of waiting for scope exit.
3. **`torch.cuda.empty_cache()`** per frame to return the now-freed blocks to the GPU.
```python
for j in range(batch_cuda.shape[0]):
input_frame = batch_cuda[j]
result = sr.run(input_frame)
dlpack_out = result.image
out_frame = torch.from_dlpack(dlpack_out).movedim(0, -1).unsqueeze(0).clone()
out_tensor[i + j: i + j + 1] = out_frame
del dlpack_out, result, out_frame
torch.cuda.empty_cache()
```
I've tested this locally: VRAM stays flat across 100+ frame runs where the original code OOM'd around ~60.
Related: #16. I'll open a PR with this fix shortly.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in __init__.py at the RTXVideoSuperResolution execute() per-frame loop and inspect the lifetime of the sr.run() result and converted tensors. Reproduce with a 60+ frame video while monitoring VRAM with nvidia-smi. Done means memory remains roughly flat through 100+ frames and the node completes without OOM or hanging.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100