Abnormal use of RAM (LTX 2.3)

### Custom Node Testing

- [ ] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Expected Behavior

LTX 2.3 loads normally and executes the prompt.

### Actual Behavior

LTX 2.3 text encoder (Gemma 3-12B-it) uses an abnormal amount of RAM to load, and doesn't wait for the text encoder swap from VRAM to the RAM completely. Then the model tries to load on RAM (or on VRAM if forced to), causing an inevitable OOM.

### Steps to Reproduce

Simply try to load LTX 2.3 and Gemma 3-12B-it on a limited RAM scenario.

[LTX GGUF (2).json](https://github.com/user-attachments/files/28528506/LTX.GGUF.2.json)

### Debug Logs

```powershell
Not really necessary because its a usage comparison.
```

### Other

I'm running ComfyUI on Google Colab (Linux), which features 12GB RAM and 16GB VRAM (T4 GPU). I use both models (model and text encoder) in the GGUF format. Everything works fine if I load the text encoder only on RAM with the node multigpu. The text encoder load peak usage reaches 8-9GB if only loaded on RAM with the node MultiGPU, while on VRAM (normal GGUF loader/no MultiGPU) it reaches 11-13GB of peak usage.
Note 1: The peak usage can be lower if you not use an abliterated version of Gemma 3-12B-it (but it stills abnormal).
Note 2: I know that this RAM limit of Google Colab sucks, but this report can be somehow helpful to optimize the load usage (I think).