LTX v0.91 is very slow after the first generation

### Expected Behavior

Generate videos always at the same speeds.

### Actual Behavior

I have an old GPU ( GTX 1070 ), but LTX 0.91, it's still usable for me: about 10 ~ 12 s/it ( in image 2 video ).

But only in the first generation.

After that it becomes very slow ( about 31 s/it ). Sometimes even slower.


### Steps to Reproduce

I used only native nodes and the example workflows.


### Debug Logs

```powershell
C:\Cui\cu_121_2\ComfyUI_windows_portable>.\python_embeded\python.exe -s ComfyUI\main.py --windows-standalone-build
Adding extra search path checkpoints d:/ComfyUI/models/checkpoints/
Adding extra search path clip d:/ComfyUI/models/clip/
Adding extra search path clip_vision d:/ComfyUI/models/clip_vision/
Adding extra search path configs d:/ComfyUI/models/configs/
Adding extra search path controlnet d:/ComfyUI/models/controlnet/
Adding extra search path embeddings d:/ComfyUI/models/embeddings/
Adding extra search path loras d:/ComfyUI/models/loras/
Adding extra search path unet d:/ComfyUI/models/unet/
Adding extra search path upscale_models d:/ComfyUI/models/upscale_models/
Adding extra search path vae d:/ComfyUI/models/vae/
[START] Security scan
[DONE] Security scan
## ComfyUI-Manager: installing dependencies done.
** ComfyUI startup time: 2025-01-03 14:59:22.668380
** Platform: Windows
** Python version: 3.11.6 (tags/v3.11.6:8b6ee5b, Oct 2 2023, 14:57:12) [MSC v.1935 64 bit (AMD64)]
** Python executable: C:\Cui\cu_121_2\ComfyUI_windows_portable\python_embeded\python.exe
** ComfyUI Path: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI
** Log path: C:\Cui\cu_121_2\ComfyUI_windows_portable\comfyui.log

[notice] A new release of pip is available: 23.3.1 -> 24.3.1
[notice] To update, run: C:\Cui\cu_121_2\ComfyUI_windows_portable\python_embeded\python.exe -m pip install --upgrade pip

[notice] A new release of pip is available: 23.3.1 -> 24.3.1
[notice] To update, run: C:\Cui\cu_121_2\ComfyUI_windows_portable\python_embeded\python.exe -m pip install --upgrade pip

Prestartup times for custom nodes:
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\rgthree-comfy
4.8 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager

Total VRAM 8192 MB, total RAM 32727 MB
pytorch version: 2.3.1+cu121
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce GTX 1070 : cudaMallocAsync
Using pytorch attention
[Prompt Server] web root: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\web
[ComfyUI-0️⃣ 2️⃣ 4️⃣ 6️⃣ ] Topological Execution is detected.
[ComfyUI-0️⃣ 2️⃣ 4️⃣ 6️⃣ ] Loaded all nodes and apis.
### Loading: ComfyUI-Impact-Pack (V8.1.6)
[Impact Pack] Wildcards loading done.
### Loading: ComfyUI-Inspire-Pack (V1.9.1)
Total VRAM 8192 MB, total RAM 32727 MB
pytorch version: 2.3.1+cu121
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce GTX 1070 : cudaMallocAsync
### Loading: ComfyUI-Manager (V2.56)
### ComfyUI Version: v0.3.10-17-g67758f5 | Released on '2024-12-31'
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
C:\Cui\cu_121_2\ComfyUI_windows_portable\python_embeded\Lib\site-packages\albumentations\__init__.py:13: UserWarning: A new version of Albumentations is available: 1.4.24 (you have 1.4.16). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1.
check_for_updates()
[ReActor] - STATUS - Running v0.5.2-a2 in ComfyUI
Torch version: 2.3.1+cu121
[comfyui_controlnet_aux] | INFO -> Using ckpts path: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux\ckpts
[comfyui_controlnet_aux] | INFO -> Using symlinks: False
[comfyui_controlnet_aux] | INFO -> Using ort providers: ['CUDAExecutionProvider', 'DirectMLExecutionProvider', 'OpenVINOExecutionProvider', 'ROCMExecutionProvider', 'CPUExecutionProvider', 'CoreMLExecutionProvider']
DWPose: Onnxruntime with acceleration providers detected
Please 'pip install xformers'
Nvidia APEX normalization not installed, using PyTorch LayerNorm

[rgthree-comfy] Loaded 42 fantastic nodes. 🎉

WAS Node Suite: BlenderNeko's Advanced CLIP Text Encode found, attempting to enable `CLIPTextEncode` support.
WAS Node Suite: `CLIPTextEncode (BlenderNeko Advanced + NSP)` node enabled under `WAS Suite/Conditioning` menu.
WAS Node Suite: OpenCV Python FFMPEG support is enabled
WAS Node Suite Warning: `ffmpeg_bin_path` is not set in `C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui\was_suite_config.json` config file. Will attempt to use system ffmpeg binaries if available.
WAS Node Suite: Finished. Loaded 221 nodes successfully.

"Believe in yourself, and the rest will fall into place." - Unknown

Import times for custom nodes:
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\cg-controller
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\websocket_image_save.py
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_Noise
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_Cutoff
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_ADV_CLIP_emb
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_TiledKSampler
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\cg-use-everywhere
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\cg-image-picker
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AutomaticCFG
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_InstantID
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_UltimateSDUpscale
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Custom-Scripts
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-GGUF
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\rgthree-comfy
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_essentials
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-0246
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\aegisflow_utility_nodes
0.0 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-KJNodes
0.1 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux
0.1 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-VideoHelperSuite
0.1 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Inspire-Pack
0.1 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus
0.1 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_smZNodes
0.1 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack
0.3 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Detail-Daemon
0.3 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\PuLID_ComfyUI
0.6 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_FaceAnalysis
0.7 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager
1.4 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui-reactor-node
2.2 seconds: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui

Starting server

To see the GUI go to: http://127.0.0.1:8188
FETCH DATA from: C:\Cui\cu_121_2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json [DONE]
got prompt
model weight dtype torch.bfloat16, manual cast: torch.float32
model_type FLUX
VAE load device: cuda:0, offload device: cpu, dtype: torch.float32
no CLIP/text encoder weights in checkpoint, the text encoder model will not be loaded.
CLIP model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load MochiTEModel_
loaded partially 5785.8 5771.38671875 0
0 models unloaded.
loaded partially 5771.386718845368 5771.38671875 0
Requested to load VideoVAE
loaded completely 9.5367431640625e+25 3566.394229888916 True
Warning: TAESD previews enabled, but could not find models/vae_approx/None
Requested to load LTXV
loaded completely 9.5367431640625e+25 3667.902587890625 True
100%|██████████████████████████████████████████████████████████████████████████████████| 25/25 [05:01<00:00, 12.08s/it]
0 models unloaded.
loaded partially 1428.9999801635743 1428.9995155334473 0
Warning: Ran out of memory when regular VAE decoding, retrying with tiled VAE decoding.
Prompt executed in 415.19 seconds
got prompt
Warning: TAESD previews enabled, but could not find models/vae_approx/None
Requested to load LTXV
loaded completely 9.5367431640625e+25 3667.902587890625 True
8%|██████▋ | 2/25 [01:54<21:54, 57.14s/it]
Processing interrupted
Prompt executed in 115.89 seconds
```

### Other