Comfy-Org / Comfy-Org/ComfyUI

RTX 5090 (Blackwell) nvfp4 bypasses PyTorch allocator causing VAE decode OOM/slowdown on 32GB VRAM

Open
#14,157 6 comments 0 reactions 1 assignee Claimed by @rattus128 View on GitHub
Potential Bug
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Custom Node Testing

- [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Expected Behavior

VAE decode completes in ~10 seconds. Continuous batch processing runs stably without slowdown or OOM errors.

### Actual Behavior

On RTX 5090 (32GB), VAE decode intermittently takes 150-300 seconds or OOM occurs during diffusion sampling on the second consecutive run. After OOM, ComfyUI unloads all models and the next run succeeds normally.

Key observation in OOM memory summary:
- Allocated memory: 26972 MiB
- Tot Alloc: 0 B ← PyTorch allocator sees nothing
- GPU reserved memory: 31584 MiB

nvfp4 native ops (enabled automatically on Blackwell/RTX 5090) allocate VRAM bypassing the PyTorch allocator. ComfyUI's memory management cannot see this memory, severely underestimates actual VRAM usage, and fails to trigger model offloading before VAE decode, leaving only 400-1100MB for VAE.

### Steps to Reproduce

1. RTX 5090 (32GB) on Windows
2. Load WAN 2.2 14B model with lightx2v 4-step LoRA (dual UNETLoader workflow)
3. Submit 2+ consecutive T2V generation tasks via API or queue
4. Observe: second run VAE decode becomes extremely slow or first run OOM

Note: Native ops log line will show "nvfp4" confirming the issue applies.

### Debug Logs

```powershell
Native ops: nvfp4, float8_e5m2, float8_e4m3fn , emulated ops: mxfp8
Requested to load WanVAE
loaded completely; 448 MB usable, 242.03 MB loaded, full load: True
[VAE decode takes 150+ seconds]

--- OOM case ---
torch.OutOfMemoryError: Allocation on device
Allocated memory: 26972 MiB | Tot Alloc: 0 B | Tot Freed: 0 B
GPU reserved memory: 31584 MiB
Got an OOM, unloading all loaded models.

--- Fix ---
Increasing EXTRA_RESERVED_VRAM in comfy/model_management.py for 16GB+ cards from +100MB to +4GB resolves the issue completely. 20+ consecutive runs stable at ~10s each.
```

### Other

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.