Lora Training node, loss value consistently exploding

### Custom Node Testing

- [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Expected Behavior

Training LoRAs using the Train LoRA beta node to work as it did before.
The loss value to remain below 1.0
The result of the LoRA training to be visually without artefacts or glitches
Train at a reasonably fast rate as the node did before the latest update (60 images, batch size 2 at around 4 seconds per iteration)

### Actual Behavior

Lora Loss explodes above 1.0:
"Training LoRA: 1%| | 1/150 [00:10<25:08, 10.12s/it, loss=1.3385]"

Resulting images from even 150 steps is a noisy mess of colour

Train LoRA node doesn't function as it did before the 0.18.0 update

Using default settings it's over a minute to do one iteration.

### Steps to Reproduce

Setup basic lora training and use settings

Using default settings also seems to just freeze the lora training completely maxing out my 6GB vram even with a single input latent image taking 1:37 to do 1 it where i was getting around 3.5 seconds before. this never used to be a problem:

### Debug Logs

```powershell
FIRST ISSUE (USING OLD TRAINING METHOD / HUGE LOSS):
got prompt
Using pytorch attention in VAE
Using pytorch attention in VAE
VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
model weight dtype torch.float16, manual cast: None
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE
VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load SD1ClipModel
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Loading images from folder: LoRA Training/********
Loaded 63 images from C:\Users\eddyg\Documents\ComfyUI\input\LoRA Training/********.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Model SD1ClipModel prepared for dynamic VRAM loading. 235MB Staged. 0 patches attached.
Requested to load AutoencoderKL
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Gradient checkpointing: patching 51 modules at depth 2
Requested to load BaseModel
0 models unloaded.
Model BaseModel prepared for dynamic VRAM loading. 1639MB Staged. 0 patches attached.
0 models unloaded.
Model BaseModel prepared for dynamic VRAM loading. 1639MB Staged. 0 patches attached.
Training LoRA: 1%| | 1/150 [00:07<19:36, 7.89s/it, loss=1.3385]Interrupting prompt 82078659-22b5-4fc5-a116-befea3118776
Training LoRA: 1%| | 1/150 [00:10<25:08, 10.12s/it, loss=1.3385]

SECOND ISSUE (FREEZING ON DEFAULT SETTINGS / OVER 1 MINUTE PER IT):
got prompt
Gradient checkpointing: patching 1 modules at depth 1
Requested to load BaseModel
0 models unloaded.
Model BaseModel prepared for dynamic VRAM loading. 1639MB Staged. 0 patches attached.
Model BaseModel prepared for dynamic VRAM loading. 1639MB Staged. 0 patches attached.
Training LoRA: 0%| | 0/150 [00:00