Enhancement on VRAM (and maybe RAM?) handling between samplers on LTX 2 comfy native workflow
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
### Custom Node Testing
- [ ] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)
### Expected Behavior
1080p above 200 frames with 25 fps video generation can be normally performed without OOM across samplers
### Actual Behavior
At 1080p above 200 frames with 25 fps video generation, when the progress move from first sampling to the second sampling (upscale 2x step), the VRAM usage shoots up and causes OOM
### Steps to Reproduce
I'm using native ComfyUI workflow (from the templates browser) I2V LTX 2 distilled workflow, nothing is changed.
The flags i'm using to run comfy ui is: `comfy launch -- --use-sage-attention --listen --novram --cache-none --disable-smart-memory --preview-method taesd`
Machine specs is: 16GB VRAM, 32GB RAM, 64GB file swap, swappiness set to 6. I'm using Linux in headless mode (no desktop environment running, operate solely using SSH, remotely)
The versions as follows (i'm on nightly channel):
At first I thought ComfyUI doesn't have room to move stuff from VRAM to RAM, so i tried also added flag `--disable-pinned-memory`, drastically reduces the RAM usage when generating, but still got OOM (VRAM shoots up when starting the second sampling).
Then I'm experimenting, to split the run between the two samplers this way:
1. Run the first sampling steps, and save all the latents to file, there is three 3 latents: audio, video before `LTXVCropGuides` node node, video after `LTXVlmgToVideolnplace` node.
2. After all latents is saved, I invert the bypassed nodes and plug all the saved latents, and it works! Successfully generated the video without OOM, the VRAM usage is at mid hovering around 50%
By that experiment, I think there's a possibly for enhancement in VRAM handling, or maybe not. So, I give it to the contributors that know the inner working of VRAM handling for LTX 2 in ComfyUI.
This issue only surface when I tried to generate 1080p video with frame above 200 frames at 25 fps. 1080p 200 frame with the same fps work just fine without the experiment.
### Debug Logs
```powershell
Requested to load LTXAV
loaded partially; 0.00 MB usable, 0.00 MB loaded, 20541.27 MB offloaded, 448.07 MB buffer reserved, lowvram patches: 0
0%| | 0/3 [00:02..., 'video_embeddings_connector.transformer_1d_blocks.1.ff.net.2.weight']
VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
no CLIP/text encoder weights in checkpoint, the text encoder model will not be loaded.
Requested to load VideoVAE
loaded partially; 0.00 MB usable, 0.00 MB loaded, 2378.23 MB offloaded, 648.02 MB buffer reserved, lowvram patches: 0
clip missing: ['gemma3_12b.logit_scale', ...... , 'gemma3_12b.transformer.vision_model.post_layernorm.bias']
Requested to load LTXAVTEModel_
loaded completely; 95367431640625005117571072.00 MB usable, 25965.49 MB loaded, full load: True
CLIP/text encoder model load device: cpu, offload device: cpu, current: cpu, dtype: torch.float16
Error running sage attention: list indices must be integers or slices, not NoneType, using pytorch attention instead.
Error running sage attention: list indices must be integers or slices, not NoneType, using pytorch attention instead.
Error running sage attention: list indices must be integers or slices, not NoneType, using pytorch attention instead.
Error running sage attention: list indices must be integers or slices, not NoneType, using pytorch attention instead.
Warning: TAESD previews enabled, but could not find models/vae_approx/None
Requested to load LTXAV
loaded partially; 0.00 MB usable, 0.00 MB loaded, 20541.27 MB offloaded, 448.07 MB buffer reserved, lowvram patches: 0
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [02:40<00:00, 20.09s/it]
Requested to load VideoVAE
loaded partially; 0.00 MB usable, 0.00 MB loaded, 2378.23 MB offloaded, 648.02 MB buffer reserved, lowvram patches: 0
VRAMdebug: free memory before: 16,481,845,248
VRAMdebug: free memory after: 16,481,845,248
VRAMdebug: freed memory: 0
Warning: TAESD previews enabled, but could not find models/vae_approx/None
```
Not sure if it's refuse to use sage attention for the text encoder (gemini) or the model itself (distilled fp8 ltx2 19B)
Contributor guide
Assessment
This issue has not been assessed yet.