Comfy-Org / Comfy-Org/ComfyUI

Node Bloat/Missing Controls/messy workflows.

Open
#11,684 1 comment 8 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 10h
Merged PRs (30d)
153

Description

The LTX 2 model implementation seems very rough.
The text encoder and VAEs should not require their own nodes to load them, not to mention the seperate nodes for audio vae decoding (LTXV Audio VAE decode). Here are the current issues regarding the current implementation of LTX 2.

1: The current implementaion is missing the control as to where to keep the Text Encoder, cpu/cuda. This should just use the cliploader node like other models.
2: Currently it loads the whole checkpoint but what if I want to generate only videos without audio? Why should my GPU load the audio parts: vocoder, audio vae, etc
3: The current implementation is much harder to maintain for both custom node devs and you guys as well.
4: No tiled VAE node for the audio VAEs
5: Extra "wires" due to the current implementation.
6: ComfyUI-GGUF and ComfyUI-MultiGPU custom nodes extend upon the cliploader/vaeloader nodes in comfyui. Now they have to add more nodes "ltxvcliploader-GGUF" "ltxvaudiovaeloader-GGUF", etc this will just lead to node bloat which will be so much more harder to maintain.
7: Duplicated files inside the CKPT like audio VAE, VAE, vocoder, etc if I have multiple models like distilled and dev
8: Having seperate model specific nodes for loading TE/VAE files for each new model will just make WFs more complex/confusing

Here are the solutions

Use the cliploader/VAE loader nodes instead of making seperate nodes to load the TE, VAEs, etc
Instead of a checkpoint, it should be repackaged like other models and the VAEs, vocoder should be seperated
Make an AIO node for audio vae decoding just like "vae decode" node.

I have 16gb VRAM and 32GB RAM, I should be able to run it without issues with fp8 TE, but it uses pagefile presumably because it's loading the gemma model onto VRAM instead of the RAM, but I don't have that control for this model because it uses it's own node for loading the TE and I have no choice to keep VAE/vocoder in RAM or load it after sampling

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.