Comfy-Org / Comfy-Org/ComfyUI

LTX models detected as FLUX type ignore FP16 flags and force torch.float32 upcast

Open
#15,338 1 comment 1 reaction 0 assignees View on GitHub
Potential Bug
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Custom Node Testing

- [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Expected Behavior

LTX models should respect --force-fp16 and --fp16-unet flags, loading with manual_cast: None instead of being forced to torch.float32 due to the hardcoded ModelType.FLUX classification.

### Actual Behavior

LTX 2.3 models (both LTXV and LTXAV) are hardcoded with model_type=ModelType.FLUX in comfy/ldm/lightricks/video_model.py. This causes ComfyUI to apply FLUX-specific dtype handling, which forces manual_cast: torch.float32 regardless of command-line flags.

model weight dtype torch.float16, manual cast: torch.float32
model_type FLUX

This happens even with --force-fp16 --fp16-unet --dont-upcast-attention flags. The FP32 upcast doubles VRAM usage and slows inference significantly. On 11 GB VRAM, this pushes memory from ~14 GB to ~26+ GB, making generation unstable or impossible.

Image

### Steps to Reproduce

Launch ComfyUI with --force-fp16 --fp16-unet
Load any LTX 2.3 checkpoint (video or audio-video)
Queue a prompt
Observe manual cast: torch.float32 in console

### Debug Logs

```powershell
D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable>.\python_embeded\python.exe -s ComfyUI\main.py --windows-standalone-build --force-fp16 --enable-triton-backend --bf16-vae --fp16-unet --fast fp16_accumulation
setup plugin alembic.autogenerate.schemas
setup plugin alembic.autogenerate.tables
setup plugin alembic.autogenerate.types
setup plugin alembic.autogenerate.constraints
setup plugin alembic.autogenerate.defaults
setup plugin alembic.autogenerate.comments
[START] Security scan
[DONE] Security scan
## ComfyUI-Manager: installing dependencies done.
** ComfyUI startup time: 2026-08-05 19:39:32.168
** Platform: Windows
** Python version: 3.12.10 (tags/v3.12.10:0cc8128, Apr 8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)]
** Python executable: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\python_embeded\python.exe
** ComfyUI Path: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\ComfyUI
** ComfyUI Base Folder Path: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\ComfyUI
** User directory: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\ComfyUI\user
** ComfyUI-Manager config path: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\ComfyUI\user\__manager\config.ini
** Log path: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\ComfyUI\user\comfyui.log

Prestartup times for custom nodes:
2.5 seconds: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager

WARNING: You need pytorch with cu130 or higher to use optimized CUDA operations.
Found triton 3.2.0. Enabling comfy-kitchen triton backend.
Found comfy_kitchen backend triton: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8']}
Found comfy_kitchen backend cuda: {'available': True, 'disabled': True, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'scaled_mm_nvfp4']}
Found comfy_kitchen backend eager: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_mxfp8', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'scaled_mm_mxfp8', 'scaled_mm_nvfp4']}
Checkpoint files will always be loaded safely.
Total VRAM 11264 MB, total RAM 40884 MB
pytorch version: 2.11.0+cu126
Enabled fp16 accumulation.
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 2080 Ti : cudaMallocAsync
Using async weight offloading with 2 streams
Enabled pinned memory 16353.0
Using pytorch attention
aimdo: src-win/cuda-detour.c:38:INFO:aimdo_setup_hooks: installing 6 hooks
aimdo: src-win/cuda-detour.c:28:DEBUG:install_hook_entries: hooks successfully installed
aimdo: src-win/shmem-detect.c:80:INFO:comfy-aimdo WDDM adapter match: NVIDIA GeForce RTX 2080 Ti runtime_luid=00000000:0000a6d9 dxgi_luid=00000000:0000a6d9
aimdo: src/control.c:152:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce RTX 2080 Ti (VRAM: 11263 MB)
DynamicVRAM support detected and enabled
Python version: 3.12.10 (tags/v3.12.10:0cc8128, Apr 8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)]
ComfyUI version: 0.22.0
comfy-aimdo version: 0.3.0
comfy-kitchen version: 0.2.8
comfyui-frontend-package version: 1.43.18
comfyui-workflow-templates version: 0.9.79
comfyui-embedded-docs version: 0.5.0
comfy-kitchen version: 0.2.8
comfy-aimdo version: 0.3.0
[Prompt Server] web root: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\python_embeded\Lib\site-packages\comfyui_frontend_package\static
Asset seeder disabled
### Loading: ComfyUI-Manager (V3.41)
[ComfyUI-Manager] network_mode: public
[ComfyUI-Manager] ComfyUI per-queue preview override detected (PR #11261). Manager's preview method feature is disabled. Use ComfyUI's --preview-method CLI option or 'Settings > Execution > Live preview method'.
### ComfyUI Revision: 150 [a8d25190] *DETACHED | Released on '2026-05-20'

Import times for custom nodes:
0.0 seconds: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\ComfyUI\custom_nodes\websocket_image_save.py
0.0 seconds: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_DiffusionModel_fp8_converter
0.0 seconds: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\ComfyUI\custom_nodes\fastvideo
0.0 seconds: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Custom-Scripts
0.0 seconds: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\ComfyUI\custom_nodes\grok-smartvae
0.0 seconds: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-bleh
0.0 seconds: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-KJNodes
0.1 seconds: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager
0.3 seconds: D:\ComfyUI_windows_portable_nvidia_cu126\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LTXVideo

Context impl SQLiteImpl.
Will assume non-transactional DDL.
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json
Starting server

To see the GUI go to: http://127.0.0.1:8188
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
[DEPRECATION WARNING] Detected import of deprecated legacy API: /scripts/ui.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
[DEPRECATION WARNING] Detected import of deprecated legacy API: /extensions/core/groupNode.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
[DEPRECATION WARNING] Detected import of deprecated legacy API: /scripts/ui/components/button.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
[DEPRECATION WARNING] Detected import of deprecated legacy API: /scripts/ui/components/buttonGroup.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
got prompt
VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
Found quantization metadata version 1
Using MixedPrecisionOps for text encoder
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load LTXAVTEModel_
Model LTXAVTEModel_ prepared for dynamic VRAM loading. 13764MB Staged. 0 patches attached. Force pre-loaded 290 weights: 1497 KB.
Found quantization metadata version 1
Detected mixed precision quantization
Using mixed precision operations
Native ops: , emulated ops: float8_e5m2, mxfp8, float8_e4m3fn, nvfp4
model weight dtype torch.float16, manual cast: torch.float32
model_type FLUX
Requested to load LTXAV
Model LTXAV prepared for dynamic VRAM loading. 13282MB Staged. 0 patches attached.
0%| | 0/8 [00:00
Traceback (most recent call last):
File "asyncio\events.py", line 88, in _run
File "asyncio\proactor_events.py", line 165, in _call_connection_lost
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle:
Traceback (most recent call last):
File "asyncio\events.py", line 88, in _run
File "asyncio\proactor_events.py", line 165, in _call_connection_lost
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
```

### Other

Workarounds attempted (none worked):
--force-fp16, --fp16-unet, --dont-upcast-attention flags
Patching unet_manual_cast() in model_management.py
Setting self.manual_cast_dtype = None in LTX class __init__
Disabling custom nodes (made it worse — memory hit 26K)

Contributor guide

Open the contributing guide

Research direction

Start in comfy/ldm/lightricks/video_model.py, where LTXV and LTXAV are classified as ModelType.FLUX, then inspect unet_manual_cast() in model_management.py. Reproduce with --force-fp16 --fp16-unet and an LTX 2.3 checkpoint, and verify that the flags no longer force manual_cast: torch.float32 or the reported VRAM increase.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
backend, machine-learning, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.