RTX 5090 (Blackwell) nvfp4 bypasses PyTorch allocator causing VAE decode OOM/slowdown on 32GB VRAM
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
### Custom Node Testing
- [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)
### Expected Behavior
VAE decode completes in ~10 seconds. Continuous batch processing runs stably without slowdown or OOM errors.
### Actual Behavior
On RTX 5090 (32GB), VAE decode intermittently takes 150-300 seconds or OOM occurs during diffusion sampling on the second consecutive run. After OOM, ComfyUI unloads all models and the next run succeeds normally.
Key observation in OOM memory summary:
- Allocated memory: 26972 MiB
- Tot Alloc: 0 B ← PyTorch allocator sees nothing
- GPU reserved memory: 31584 MiB
nvfp4 native ops (enabled automatically on Blackwell/RTX 5090) allocate VRAM bypassing the PyTorch allocator. ComfyUI's memory management cannot see this memory, severely underestimates actual VRAM usage, and fails to trigger model offloading before VAE decode, leaving only 400-1100MB for VAE.
### Steps to Reproduce
1. RTX 5090 (32GB) on Windows
2. Load WAN 2.2 14B model with lightx2v 4-step LoRA (dual UNETLoader workflow)
3. Submit 2+ consecutive T2V generation tasks via API or queue
4. Observe: second run VAE decode becomes extremely slow or first run OOM
Note: Native ops log line will show "nvfp4" confirming the issue applies.
### Debug Logs
```powershell
Native ops: nvfp4, float8_e5m2, float8_e4m3fn , emulated ops: mxfp8
Requested to load WanVAE
loaded completely; 448 MB usable, 242.03 MB loaded, full load: True
[VAE decode takes 150+ seconds]
--- OOM case ---
torch.OutOfMemoryError: Allocation on device
Allocated memory: 26972 MiB | Tot Alloc: 0 B | Tot Freed: 0 B
GPU reserved memory: 31584 MiB
Got an OOM, unloading all loaded models.
--- Fix ---
Increasing EXTRA_RESERVED_VRAM in comfy/model_management.py for 16GB+ cards from +100MB to +4GB resolves the issue completely. 20+ consecutive runs stable at ~10s each.
```
### Other
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.