Extreme decrease in performance update from August 2024

### Expected Behavior

generate an image in 30-40 secs on my machine

### Actual Behavior

Since I updated to the new interface it takes a very long time (~7 min.) to generate an image. Front end logged issue #2084 denies front end implications

### Steps to Reproduce

launch with python main.py from terminal

### Debug Logs

```powershell
** ComfyUI start up time: 2024-12-28 15:26:52.068607

Prestartup times for custom nodes:
0.0 seconds: /home/comfy/Downloads/ComfyUI/custom_nodes/ComfyUI-Manager

Total VRAM 5751 MB, total RAM 64104 MB
pytorch version: 2.3.0+cu121
xformers version: 0.0.26.post1
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce GTX 1660 Ti with Max-Q Design : cudaMallocAsync
Using xformers attention
[Prompt Server] web root: /home/comfy/Downloads/ComfyUI/web
[comfyui_controlnet_aux] | INFO -> Using ckpts path: /home/comfy/Downloads/ComfyUI/custom_nodes/comfyui_controlnet_aux/ckpts
[comfyui_controlnet_aux] | INFO -> Using symlinks: False
[comfyui_controlnet_aux] | INFO -> Using ort providers: ['CUDAExecutionProvider', 'DirectMLExecutionProvider', 'OpenVINOExecutionProvider', 'ROCMExecutionProvider', 'CPUExecutionProvider', 'CoreMLExecutionProvider']
DWPose: Onnxruntime with acceleration providers detected
Adding /home/comfy/Downloads/ComfyUI/custom_nodes to sys.path
Could not find efficiency nodes
Loaded ControlNetPreprocessors nodes from /home/comfy/Downloads/ComfyUI/custom_nodes/comfyui_controlnet_aux
Could not find AdvancedControlNet nodes
Could not find AnimateDiff nodes
Could not find IPAdapter nodes
Could not find VideoHelperSuite nodes
Could not load ImpactPack nodes Could not find ImpactPack nodes
loading image tools...
### Loading: ComfyUI-Manager (V0.26.2)
### ComfyUI Revision: 2981 [c4bfdba3]

Import times for custom nodes:
0.0 seconds: /home/comfy/Downloads/ComfyUI/custom_nodes/websocket_image_save.py
0.0 seconds: /home/comfy/Downloads/ComfyUI/custom_nodes/ComfyUI-Inpaint-CropAndStitch
0.0 seconds: /home/comfy/Downloads/ComfyUI/custom_nodes/comfyui-fitsize
0.0 seconds: /home/comfy/Downloads/ComfyUI/custom_nodes/comfyui-various
0.0 seconds: /home/comfy/Downloads/ComfyUI/custom_nodes/comfyui_met_suite
0.0 seconds: /home/comfy/Downloads/ComfyUI/custom_nodes/cg-use-everywhere
0.0 seconds: /home/comfy/Downloads/ComfyUI/custom_nodes/image-resize-comfyui
0.7 seconds: /home/comfy/Downloads/ComfyUI/custom_nodes/comfyui-art-venture
1.0 seconds: /home/comfy/Downloads/ComfyUI/custom_nodes/ComfyUI-Manager
1.2 seconds: /home/comfy/Downloads/ComfyUI/custom_nodes/comfyui_segment_anything
4.5 seconds: /home/comfy/Downloads/ComfyUI/custom_nodes/ComfyUI-Image-Tools
4.9 seconds: /home/comfy/Downloads/ComfyUI/custom_nodes/comfyui_controlnet_aux

Starting server

To see the GUI go to: http://127.0.0.1:8188
FETCH DATA from: /home/comfy/Downloads/ComfyUI/custom_nodes/ComfyUI-Manager/extension-node-map.json
got prompt
model weight dtype torch.bfloat16, manual cast: torch.float32
model_type EPS
Using xformers attention in VAE
Using xformers attention in VAE
VAE load device: cuda:0, offload device: cpu, dtype: torch.float32
Requested to load SDXLClipModel
loaded completely 9.5367431640625e+25 1560.802734375 True
CLIP model load device: cpu, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load AutoencoderKL
0 models unloaded.
loaded completely 9.5367431640625e+25 319.11416244506836 True
Warning: Ran out of memory when regular VAE encoding, retrying with tiled VAE encoding.
Requested to load SDXL
loaded partially 2063.346744415283 2063.345947265625 0
100%|##########| 20/20 [16:31<00:00, 49.56s/it]
Requested to load AutoencoderKL
0 models unloaded.
loaded completely 9.5367431640625e+25 319.11416244506836 True
Warning: Ran out of memory when regular VAE decoding, retrying with tiled VAE decoding.
Prompt executed in 1269.76 seconds
got prompt
```

### Other

Brave browser