Comfy-Org / Comfy-Org/ComfyUI

MiniMax H3 video VAE: long-clip decode OOMs are unrecoverable — tiled retry is a no-op and dynamic VRAM never evicts for the decode

Open
#15,453 2 comments 3 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Summary

On a 16GB card, MiniMax H3 video generation above ~209 frames reliably completes sampling and then dies in `VAEDecode`, after paying the full multi-minute sampling cost. Two independent problems combine to make this unrecoverable:

1. **The tiled-decode fallback is a no-op for this VAE.** `MiniMaxH3VideoVAE.decode_tiled()` just calls `decode()` (`comfy/ldm/minimax/vae.py`, the model tiles internally and the VAE sets `handles_tiling = True`), so the "Ran out of memory when regular VAE decoding, retrying with tiled VAE decoding" path in `comfy/sd.py` re-runs the identical decode and fails identically.
2. **Under dynamic VRAM, nothing is evicted for the decode.** At the point of failure ~14GB of dynamically streamed weights (DiT + text encoder) sit outside the torch allocator, `load_models_gpu` for the VAE reports `0 models unloaded`, and the decode's ~1.1–1.2GiB peak torch allocations fail with ~0.8GiB free. Raising the VAE's `memory_used_decode` estimate has **no effect** — the declared requirement is never converted into eviction on the dynamic path. Free VRAM at failure was also *below* `--reserve-vram 1.5`, so the reserve does not bound streamed residency either.

### Environment

- ComfyUI v0.30.2 (`dec5d945`), torch 2.12.1 cu130, `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True`
- RTX Pro 2000 Blackwell 16GB (15.52 GiB usable, sm_120), driver 610.43.02 open modules, Debian 12 / Docker
- Flags: `--fast --use-sage-attention --reserve-vram 1.5 --preview-method auto`
- Models: `minimax_h3_fl2va_pruned_nvfp4.safetensors` (12.5GB DiT), `qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors` encoder, stock H3 video VAE fp16 + audio VAE fp32

### Repro

Standard H3 t2v graph (`MiniMaxH3ImageToVideo` sans image / `EmptyMiniMaxH3LatentAV` path, `BasicGuider` → `SamplerCustomAdvanced`, res_multistep, 10 steps) at **864×480, `length` 243**, then `VAEDecode` + `VAEDecodeAudio` → `CreateVideo`. Sampling completes (~3:40 at ~24 s/it); decode then:

```
Requested to load MiniMaxH3VideoVAE
0 models unloaded.
Model MiniMaxH3VideoVAE prepared for dynamic VRAM loading. 4965MB Staged. ...
Warning: Ran out of memory when regular VAE decoding, retrying with tiled VAE decoding.
!!! Exception during processing !!! CUDA out of memory. Tried to allocate 1.13 GiB.
GPU 0 has a total capacity of 15.52 GiB of which 821.56 MiB is free. Including
non-PyTorch memory, this process has 14.67 GiB memory in use. Of the allocated
memory 695.74 MiB is allocated by PyTorch, and 32.26 MiB is reserved by PyTorch
but unallocated.
Got an OOM, unloading all loaded models.
Prompt executed in 366.40 seconds
```

Reproduced 3/3 at 243f. At 362f it *sometimes* survives — when the first OOM's cleanup happens to release enough for the retry — which is a race, not a recovery path.

Note the telling number: only ~0.7GiB of the 14.67GiB in use is PyTorch-allocated. The rest is dynamically streamed weight memory the failing torch allocation cannot reclaim.

### Diagnosis confirmation

Inserting a graph-level unload between the sampler and both decode nodes (a passthrough node calling `comfy.model_management.unload_all_models()`) makes the *identical* config complete with **regular** (non-tiled) decode: 243f finishes in 302s, no warning, repeatedly. So the memory exists; it is just never released for the decode.

I also tried raising the H3 branch's `estimate_decode_memory` coefficient in `comfy/sd.py` (9.5 → 20.0, i.e. declaring ~5.2GB at 243f instead of ~2.5GB): no behavioral change under dynamic VRAM — `0 models unloaded` either way. (The 9.5 coefficient does look low against the observed working set regardless, but fixing it alone is not sufficient.)

### Suggested directions

- In `sd.py`'s decode OOM handler, actually free memory before the retry (`model_management.free_memory(...)` / release streamed mappings), instead of — for `handles_tiling` VAEs — re-running the same call.
- More generally: when a torch allocation OOMs while dynamic VRAM holds streamed weights, release streamed memory and retry. The DiT is dead weight during decode; evicting it costs a few seconds of re-staging on the next prompt.
- Honor `memory_used_decode` (and `--reserve-vram`) on the dynamic-VRAM path, so the decode's declared requirement translates into headroom before the first attempt.

Happy to provide full logs, the exact API-format graph, or to test patches — this box hits the failure deterministically.

Contributor guide

Open the contributing guide

Research direction

Start with the VAEDecode OOM handler in comfy/sd.py and MiniMaxH3VideoVAE.decode_tiled() in comfy/ldm/minimax/vae.py, then trace dynamic loading through model_management and load_models_gpu. Reproduce the 864×480, length-243 graph and compare behavior with the reported unload step. Done means long-clip decode recovers without manual unloading or an identical no-op retry, while preserving normal decode behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
ai, backend, machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.