Starting to get OOM issues with my workflows

### Expected Behavior

These are workflows that have not stopped with OOM errors in earlier versions.

### Actual Behavior

with various existing workflows, it has started reporting allocation errors.
For this execution, it ran several times producing finished generations but stopped suddenly on facedetailer, see logs.

### Steps to Reproduce

Not sure if easily reproduced as it successfully generated several images before stopping. I have attached the workflow. I have the temp image that it was working on when it crashed if you like I can upload it.

[error-workflow.json](https://github.com/user-attachments/files/16913729/error-workflow.json)

### Debug Logs

```powershell
# ComfyUI Error Report
## Error Details
- **Node Type:** FaceDetailerPipe
- **Exception Type:** torch.OutOfMemoryError
- **Exception Message:** Allocation on device
## Stack Trace

File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\execution.py", line 317, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\execution.py", line 192, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)

File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\impact_pack.py", line 1381, in doit
enhanced_img, cropped_enhanced, cropped_enhanced_alpha, mask, cnet_pil_list = FaceDetailer.enhance_face(
^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\impact_pack.py", line 518, in enhance_face
sam_mask = core.make_sam_mask(sam_model_opt, segs, image, sam_detection_hint, sam_dilation,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\core.py", line 707, in make_sam_mask
detected_masks = sam_obj.predict(image, points, plabs, dilated_bbox, threshold)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\core.py", line 578, in predict
predictor.set_image(image, "RGB")

File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\segment_anything\predictor.py", line 60, in set_image
self.set_torch_image(input_image_torch, image.shape[:2])

File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils\_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\segment_anything\predictor.py", line 89, in set_torch_image
self.features = self.model.image_encoder(input_image)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\segment_anything\modeling\image_encoder.py", line 112, in forward
x = blk(x)
^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\segment_anything\modeling\image_encoder.py", line 174, in forward
x = self.attn(x)
^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\segment_anything\modeling\image_encoder.py", line 234, in forward
attn = add_decomposed_rel_pos(attn, q, self.rel_pos_h, self.rel_pos_w, (H, W), (H, W))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\segment_anything\modeling\image_encoder.py", line 358, in add_decomposed_rel_pos
attn.view(B, q_h, q_w, k_h, k_w) + rel_h[:, :, :, :, None] + rel_w[:, :, :, None, :]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~

```
## System Information
- **ComfyUI Version:** v0.2.2
- **Arguments:** ComfyUI\main.py --windows-standalone-build
- **OS:** nt
- **Python Version:** 3.11.6 (tags/v3.11.6:8b6ee5b, Oct 2 2023, 14:57:12) [MSC v.1935 64 bit (AMD64)]
- **Embedded Python:** true
- **PyTorch Version:** 2.4.1+cu121
## Devices

- **Name:** cuda:0 NVIDIA GeForce RTX 3060 : cudaMallocAsync
- **Type:** cuda
- **VRAM Total:** 12884377600
- **VRAM Free:** 5426666528
- **Torch VRAM Total:** 7247757312
- **Torch VRAM Free:** 950295584

## Logs
```
2024-09-07 03:19:15,436 - root - INFO - Unloading models for lowram load.
2024-09-07 03:19:15,447 - root - INFO - 0 models unloaded.
2024-09-07 03:19:16,066 - root - INFO - loaded partially 3753.1017768859865 3753.1005935668945 0
2024-09-07 03:19:16,688 - root - INFO - loaded partially 3753.1017768859865 3753.1005935668945 0
2024-09-07 03:19:17,304 - root - INFO - loaded partially 3753.1135261535646 3753.1128005981445 0
2024-09-07 03:19:17,993 - root - INFO - loaded partially 3753.1530464172365 3753.1518630981445 0
2024-09-07 03:19:18,604 - root - INFO - Unloading models for lowram load.
2024-09-07 03:19:18,616 - root - INFO - 0 models unloaded.
2024-09-07 03:19:19,197 - root - INFO - loaded partially 3753.1408393859865 3753.1396560668945 0
2024-09-07 03:19:19,817 - root - INFO - loaded partially 3753.1408393859865 3753.1396560668945 0
2024-09-07 03:19:20,401 - root - INFO - loaded partially 3753.1408393859865 3753.1396560668945 0
2024-09-07 03:19:21,034 - root - INFO - loaded partially 3753.1408393859865 3753.1396560668945 0
2024-09-07 03:19:21,685 - root - INFO - loaded partially 3753.1408393859865 3753.1396560668945 0
2024-09-07 03:19:22,338 - root - INFO - loaded partially 3753.1408393859865 3753.1396560668945 0
2024-09-07 03:19:22,932 - root - INFO - loaded partially 3753.1408393859865 3753.1396560668945 0
2024-09-07 03:19:23,516 - root - INFO - loaded partially 3753.1408393859865 3753.1396560668945 0
2024-09-07 03:19:24,107 - root - INFO - loaded partially 3753.1432807922365 3753.1420974731445 0
2024-09-07 03:19:24,738 - root - INFO - loaded partially 3753.1564643859865 3753.1555252075195 0
2024-09-07 03:19:25,306 - root - INFO - Unloading models for lowram load.
2024-09-07 03:19:25,318 - root - INFO - 0 models unloaded.
2024-09-07 03:19:25,997 - root - INFO - loaded partially 3753.0524604797365 3753.0517654418945 0
2024-09-07 03:19:26,706 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:27,410 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:28,087 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:28,762 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:29,435 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:30,143 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:30,820 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:31,493 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:32,204 - root - INFO - loaded partially 3753.1252143859865 3753.1250076293945 0
2024-09-07 03:19:32,781 - root - INFO - Unloading models for lowram load.
2024-09-07 03:19:32,792 - root - INFO - 0 models unloaded.
2024-09-07 03:19:33,415 - root - INFO - Unloading models for lowram load.
2024-09-07 03:19:33,426 - root - INFO - 0 models unloaded.
2024-09-07 03:19:34,107 - root - INFO - loaded partially 3753.0524604797365 3753.0517654418945 0
2024-09-07 03:19:34,782 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:35,460 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:36,135 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:36,808 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:37,486 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:38,191 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:38,929 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:39,651 - root - INFO - loaded partially 3753.0627143859865 3753.0615310668945 0
2024-09-07 03:19:40,388 - root - INFO - loaded partially 3753.1540229797365 3753.1530838012695 0
2024-09-07 03:19:40,923 - root - INFO - Unloading models for lowram load.
2024-09-07 03:19:40,935 - root - INFO - 0 models unloaded.
2024-09-07 03:19:41,667 - root - INFO - loaded partially 3753.0524604797365 3753.0517654418945 0
2024-09-07 03:19:42,378 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:43,059 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:43,733 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:44,442 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:45,115 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:45,789 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:46,494 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:47,203 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:47,912 - root - INFO - loaded partially 3753.1252143859865 3753.1250076293945 0
2024-09-07 03:19:48,491 - root - INFO - Unloading models for lowram load.
2024-09-07 03:19:48,503 - root - INFO - 0 models unloaded.
2024-09-07 03:19:49,113 - root - INFO - Unloading models for lowram load.
2024-09-07 03:19:49,125 - root - INFO - 0 models unloaded.
2024-09-07 03:19:49,801 - root - INFO - loaded partially 3753.0524604797365 3753.0517654418945 0
2024-09-07 03:19:50,478 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:51,153 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:51,828 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:52,527 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:53,200 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:53,877 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:54,585 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:55,261 - root - INFO - loaded partially 3753.0627143859865 3753.0615310668945 0
2024-09-07 03:19:55,961 - root - INFO - loaded partially 3753.1540229797365 3753.1530838012695 0
2024-09-07 03:19:56,499 - root - INFO - Unloading models for lowram load.
2024-09-07 03:19:56,512 - root - INFO - 0 models unloaded.
2024-09-07 03:19:57,185 - root - INFO - loaded partially 3753.0524604797365 3753.0517654418945 0
2024-09-07 03:19:57,897 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:58,588 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:19:59,327 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:20:00,003 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:20:00,681 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:20:01,354 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:20:02,031 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:20:02,751 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:20:03,426 - root - INFO - loaded partially 3753.1252143859865 3753.1250076293945 0
2024-09-07 03:20:04,011 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:04,022 - root - INFO - 0 models unloaded.
2024-09-07 03:20:04,617 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:04,629 - root - INFO - 0 models unloaded.
2024-09-07 03:20:05,253 - root - INFO - loaded partially 3753.0915229797365 3753.0908279418945 0
2024-09-07 03:20:05,898 - root - INFO - loaded partially 3753.0939643859865 3753.0932693481445 0
2024-09-07 03:20:06,554 - root - INFO - loaded partially 3753.0939643859865 3753.0932693481445 0
2024-09-07 03:20:07,183 - root - INFO - loaded partially 3753.0939643859865 3753.0932693481445 0
2024-09-07 03:20:07,810 - root - INFO - loaded partially 3753.0939643859865 3753.0932693481445 0
2024-09-07 03:20:08,437 - root - INFO - loaded partially 3753.0939643859865 3753.0932693481445 0
2024-09-07 03:20:09,064 - root - INFO - loaded partially 3753.0939643859865 3753.0932693481445 0
2024-09-07 03:20:09,696 - root - INFO - loaded partially 3753.0939643859865 3753.0932693481445 0
2024-09-07 03:20:10,322 - root - INFO - loaded partially 3753.0993354797365 3753.0981521606445 0
2024-09-07 03:20:10,963 - root - INFO - loaded partially 3753.1762397766115 3753.1750564575195 0
2024-09-07 03:20:11,515 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:11,527 - root - INFO - 0 models unloaded.
2024-09-07 03:20:12,162 - root - INFO - loaded partially 3753.1305854797365 3753.1298904418945 0
2024-09-07 03:20:12,752 - root - INFO - loaded partially 3753.1330268859865 3753.1323318481445 0
2024-09-07 03:20:13,336 - root - INFO - loaded partially 3753.1330268859865 3753.1323318481445 0
2024-09-07 03:20:13,933 - root - INFO - loaded partially 3753.1330268859865 3753.1323318481445 0
2024-09-07 03:20:14,518 - root - INFO - loaded partially 3753.1366889953615 3753.1359939575195 0
2024-09-07 03:20:15,131 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:15,143 - root - INFO - 0 models unloaded.
2024-09-07 03:20:15,742 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:15,754 - root - INFO - 0 models unloaded.
2024-09-07 03:20:16,438 - root - INFO - loaded partially 3753.0524604797365 3753.0517654418945 0
2024-09-07 03:20:17,114 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:20:17,785 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:20:18,464 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:20:19,176 - root - INFO - loaded partially 3753.1271675109865 3753.1262283325195 0
2024-09-07 03:20:19,832 - root - INFO - loaded partially 3753.1676948547365 3753.1665115356445 0
2024-09-07 03:20:20,352 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:20,367 - root - INFO - 0 models unloaded.
2024-09-07 03:20:21,107 - root - INFO - loaded partially 3753.0536811828615 3753.0529861450195 0
2024-09-07 03:20:21,843 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:20:22,567 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:22,577 - root - INFO - 0 models unloaded.
2024-09-07 03:20:23,285 - root - INFO - loaded partially 3753.0536811828615 3753.0529861450195 0
2024-09-07 03:20:23,956 - root - INFO - loaded partially 3753.0622261047365 3753.0615310668945 0
2024-09-07 03:20:24,667 - root - INFO - loaded partially 3753.1039741516115 3753.1030349731445 0
2024-09-07 03:20:25,275 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:25,287 - root - INFO - 0 models unloaded.
2024-09-07 03:20:25,990 - root - INFO - loaded partially 3753.0536811828615 3753.0529861450195 0
2024-09-07 03:20:26,666 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:20:27,339 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:27,349 - root - INFO - 0 models unloaded.
2024-09-07 03:20:28,088 - root - INFO - loaded partially 3753.0536811828615 3753.0529861450195 0
2024-09-07 03:20:28,761 - root - INFO - loaded partially 3753.1247261047365 3753.1237869262695 0
2024-09-07 03:20:29,354 - root - INFO - loaded partially 3753.1664741516115 3753.1652908325195 0
2024-09-07 03:20:29,870 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:29,883 - root - INFO - 0 models unloaded.
2024-09-07 03:20:30,558 - root - INFO - loaded partially 3753.0487983703615 3753.0481033325195 0
2024-09-07 03:20:31,277 - root - INFO - loaded partially 3753.0500190734865 3753.0493240356445 0
2024-09-07 03:20:31,934 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:31,945 - root - INFO - 0 models unloaded.
2024-09-07 03:20:32,664 - root - INFO - loaded partially 3753.0487983703615 3753.0481033325195 0
2024-09-07 03:20:33,339 - root - INFO - loaded partially 3753.0573432922365 3753.0566482543945 0
2024-09-07 03:20:34,008 - root - INFO - loaded partially 3753.1039741516115 3753.1030349731445 0
2024-09-07 03:20:34,611 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:34,623 - root - INFO - 0 models unloaded.
2024-09-07 03:20:35,299 - root - INFO - loaded partially 3753.0536811828615 3753.0529861450195 0
2024-09-07 03:20:35,970 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:20:36,629 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:36,640 - root - INFO - 0 models unloaded.
2024-09-07 03:20:37,315 - root - INFO - loaded partially 3753.0536811828615 3753.0529861450195 0
2024-09-07 03:20:37,994 - root - INFO - loaded partially 3753.1247261047365 3753.1237869262695 0
2024-09-07 03:20:38,619 - root - INFO - loaded partially 3753.1664741516115 3753.1652908325195 0
2024-09-07 03:20:39,135 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:39,150 - root - INFO - 0 models unloaded.
2024-09-07 03:20:39,840 - root - INFO - loaded partially 3753.0487983703615 3753.0481033325195 0
2024-09-07 03:20:40,526 - root - INFO - loaded partially 3753.0500190734865 3753.0493240356445 0
2024-09-07 03:20:41,248 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:41,258 - root - INFO - 0 models unloaded.
2024-09-07 03:20:41,999 - root - INFO - loaded partially 3753.0487983703615 3753.0481033325195 0
2024-09-07 03:20:42,703 - root - INFO - loaded partially 3753.0573432922365 3753.0566482543945 0
2024-09-07 03:20:43,403 - root - INFO - loaded partially 3753.1039741516115 3753.1030349731445 0
2024-09-07 03:20:44,036 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:44,048 - root - INFO - 0 models unloaded.
2024-09-07 03:20:44,739 - root - INFO - loaded partially 3753.0536811828615 3753.0529861450195 0
2024-09-07 03:20:45,447 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:20:46,135 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:46,145 - root - INFO - 0 models unloaded.
2024-09-07 03:20:46,837 - root - INFO - loaded partially 3753.0536811828615 3753.0529861450195 0
2024-09-07 03:20:47,516 - root - INFO - loaded partially 3753.1247261047365 3753.1237869262695 0
2024-09-07 03:20:48,173 - root - INFO - loaded partially 3753.1664741516115 3753.1652908325195 0
2024-09-07 03:20:48,693 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:48,706 - root - INFO - 0 models unloaded.
2024-09-07 03:20:49,379 - root - INFO - loaded partially 3753.0487983703615 3753.0481033325195 0
2024-09-07 03:20:50,055 - root - INFO - loaded partially 3753.0500190734865 3753.0493240356445 0
2024-09-07 03:20:50,715 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:50,725 - root - INFO - 0 models unloaded.
2024-09-07 03:20:51,397 - root - INFO - loaded partially 3753.0487983703615 3753.0481033325195 0
2024-09-07 03:20:52,119 - root - INFO - loaded partially 3753.0573432922365 3753.0566482543945 0
2024-09-07 03:20:52,789 - root - INFO - loaded partially 3753.1039741516115 3753.1030349731445 0
2024-09-07 03:20:53,387 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:53,399 - root - INFO - 0 models unloaded.
2024-09-07 03:20:54,076 - root - INFO - loaded partially 3753.0536811828615 3753.0529861450195 0
2024-09-07 03:20:54,774 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:20:55,430 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:55,441 - root - INFO - 0 models unloaded.
2024-09-07 03:20:56,135 - root - INFO - loaded partially 3753.0536811828615 3753.0529861450195 0
2024-09-07 03:20:56,813 - root - INFO - loaded partially 3753.1247261047365 3753.1237869262695 0
2024-09-07 03:20:57,403 - root - INFO - loaded partially 3753.1664741516115 3753.1652908325195 0
2024-09-07 03:20:57,928 - root - INFO - Unloading models for lowram load.
2024-09-07 03:20:57,943 - root - INFO - 0 models unloaded.
2024-09-07 03:20:58,680 - root - INFO - loaded partially 3753.0487983703615 3753.0481033325195 0
2024-09-07 03:20:59,362 - root - INFO - loaded partially 3753.0500190734865 3753.0493240356445 0
2024-09-07 03:21:00,044 - root - INFO - Unloading models for lowram load.
2024-09-07 03:21:00,054 - root - INFO - 0 models unloaded.
2024-09-07 03:21:00,790 - root - INFO - loaded partially 3753.0487983703615 3753.0481033325195 0
2024-09-07 03:21:01,529 - root - INFO - loaded partially 3753.0573432922365 3753.0566482543945 0
2024-09-07 03:21:02,229 - root - INFO - loaded partially 3753.1581733703615 3753.1579666137695 0
2024-09-07 03:21:02,780 - root - INFO - Unloading models for lowram load.
2024-09-07 03:21:02,791 - root - INFO - 0 models unloaded.
2024-09-07 03:21:03,346 - root - INFO - loaded partially 3753.1449897766115 3753.1445388793945 0
2024-09-07 03:21:03,934 - root - INFO - loaded partially 3753.1486518859865 3753.1482009887695 0
2024-09-07 03:21:04,495 - root - INFO - loaded partially 3753.1486518859865 3753.1482009887695 0
2024-09-07 03:21:05,084 - root - INFO - loaded partially 3753.1486518859865 3753.1482009887695 0
2024-09-07 03:21:05,682 - root - INFO - loaded partially 3753.1667182922365 3753.1665115356445 0
2024-09-07 03:21:06,215 - root - INFO - Unloading models for lowram load.
2024-09-07 03:21:06,227 - root - INFO - 0 models unloaded.
2024-09-07 03:21:06,895 - root - INFO - Unloading models for lowram load.
2024-09-07 03:21:06,906 - root - INFO - 0 models unloaded.
2024-09-07 03:21:07,593 - root - INFO - loaded partially 3753.0824897766115 3753.0822830200195 0
2024-09-07 03:21:08,215 - root - INFO - Unloading models for lowram load.
2024-09-07 03:21:08,226 - root - INFO - 0 models unloaded.
2024-09-07 03:21:08,861 - root - INFO - loaded partially 3753.0812690734865 3753.0810623168945 0
2024-09-07 03:21:09,500 - root - INFO - loaded partially 3753.0812690734865 3753.0810623168945 0
2024-09-07 03:21:10,151 - root - INFO - loaded partially 3753.0812690734865 3753.0810623168945 0
2024-09-07 03:21:10,787 - root - INFO - loaded partially 3753.0812690734865 3753.0810623168945 0
2024-09-07 03:21:11,425 - root - INFO - loaded partially 3753.0812690734865 3753.0810623168945 0
2024-09-07 03:21:12,076 - root - INFO - loaded partially 3753.0812690734865 3753.0810623168945 0
2024-09-07 03:21:12,760 - root - INFO - loaded partially 3753.0812690734865 3753.0810623168945 0
2024-09-07 03:21:13,401 - root - INFO - loaded partially 3753.1633003234865 3753.1628494262695 0
2024-09-07 03:21:13,933 - root - INFO - Unloading models for lowram load.
2024-09-07 03:21:13,946 - root - INFO - 0 models unloaded.
2024-09-07 03:21:14,623 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:15,306 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:15,980 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:16,656 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:17,330 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:18,003 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:18,678 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:19,417 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:20,155 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:20,890 - root - INFO - loaded partially 3753.1017768859865 3753.1005935668945 0
2024-09-07 03:21:21,514 - root - INFO - loaded partially 3753.1149604797365 3753.1140213012695 0
2024-09-07 03:21:22,160 - root - INFO - Unloading models for lowram load.
2024-09-07 03:21:22,172 - root - INFO - 0 models unloaded.
2024-09-07 03:21:22,847 - root - INFO - loaded partially 3753.0524604797365 3753.0517654418945 0
2024-09-07 03:21:23,532 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:24,235 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:24,914 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:25,623 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:26,296 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:27,003 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:27,679 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:28,351 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:29,038 - root - INFO - loaded partially 3753.1642768859865 3753.1640701293945 0
2024-09-07 03:21:29,571 - root - INFO - Unloading models for lowram load.
2024-09-07 03:21:29,584 - root - INFO - 0 models unloaded.
2024-09-07 03:21:30,256 - root - INFO - loaded partially 3753.0524604797365 3753.0517654418945 0
2024-09-07 03:21:30,951 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:31,628 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:32,301 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:33,000 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:33,690 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:34,364 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:35,042 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:35,719 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:36,392 - root - INFO - loaded partially 3753.1017768859865 3753.1005935668945 0
2024-09-07 03:21:37,017 - root - INFO - loaded partially 3753.1149604797365 3753.1140213012695 0
2024-09-07 03:21:37,600 - root - INFO - Unloading models for lowram load.
2024-09-07 03:21:37,616 - root - INFO - 0 models unloaded.
2024-09-07 03:21:38,289 - root - INFO - loaded partially 3753.0524604797365 3753.0517654418945 0
2024-09-07 03:21:38,966 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:39,706 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:40,417 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:41,118 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:41,800 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:42,473 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:43,161 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:43,869 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:44,544 - root - INFO - loaded partially 3753.1642768859865 3753.1640701293945 0
2024-09-07 03:21:45,066 - root - INFO - Unloading models for lowram load.
2024-09-07 03:21:45,080 - root - INFO - 0 models unloaded.
2024-09-07 03:21:45,789 - root - INFO - loaded partially 3753.0524604797365 3753.0517654418945 0
2024-09-07 03:21:46,464 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:47,186 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:47,863 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:48,538 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:49,215 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:49,897 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:50,581 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:51,259 - root - INFO - loaded partially 3753.0549018859865 3753.0542068481445 0
2024-09-07 03:21:51,994 - root - INFO - loaded partially 3753.1017768859865 3753.1005935668945 0
2024-09-07 03:21:52,616 - root - INFO - loaded partially 3753.1618354797365 3753.1616287231445 0
2024-09-07 03:21:53,192 - root - INFO - Unloading models for lowram load.
2024-09-07 03:21:53,203 - root - INFO - 0 models unloaded.
2024-09-07 03:21:53,758 - root - INFO - loaded partially 3753.1486518859865 3753.1482009887695 0
2024-09-07 03:21:54,314 - root - INFO - loaded partially 3753.1486518859865 3753.1482009887695 0
2024-09-07 03:21:54,874 - root - INFO - loaded partially 3753.1486518859865 3753.1482009887695 0
2024-09-07 03:21:55,440 - root - INFO - loaded partially 3753.1486518859865 3753.1482009887695 0
2024-09-07 03:21:56,008 - root - INFO - loaded partially 3753.1486518859865 3753.1482009887695 0
2024-09-07 03:21:56,604 - root - INFO - loaded partially 3753.1486518859865 3753.1482009887695 0
2024-09-07 03:21:57,171 - root - INFO - loaded partially 3753.1486518859865 3753.1482009887695 0
2024-09-07 03:21:57,737 - root - INFO - loaded partially 3753.1486518859865 3753.1482009887695 0
2024-09-07 03:21:58,323 - root - INFO - loaded partially 3753.1486518859865 3753.1482009887695 0
2024-09-07 03:21:58,940 - root - INFO - loaded partially 3753.1759956359865 3753.1750564575195 0
2024-09-07 03:21:59,511 - root - INFO - Requested to load AutoencoderKL
2024-09-07 03:21:59,511 - root - INFO - Loading 1 new model
2024-09-07 03:21:59,570 - root - INFO - loaded completely 0.0 159.55708122253418 True
2024-09-07 03:23:05,484 - root - ERROR - !!! Exception during processing !!! Allocation on device
2024-09-07 03:23:05,524 - root - ERROR - Traceback (most recent call last):
File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\execution.py", line 317, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\execution.py", line 192, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)
File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\impact_pack.py", line 1381, in doit
enhanced_img, cropped_enhanced, cropped_enhanced_alpha, mask, cnet_pil_list = FaceDetailer.enhance_face(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\impact_pack.py", line 518, in enhance_face
sam_mask = core.make_sam_mask(sam_model_opt, segs, image, sam_detection_hint, sam_dilation,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\core.py", line 707, in make_sam_mask
detected_masks = sam_obj.predict(image, points, plabs, dilated_bbox, threshold)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\core.py", line 578, in predict
predictor.set_image(image, "RGB")
File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\segment_anything\predictor.py", line 60, in set_image
self.set_torch_image(input_image_torch, image.shape[:2])
File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils\_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\segment_anything\predictor.py", line 89, in set_torch_image
self.features = self.model.image_encoder(input_image)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\segment_anything\modeling\image_encoder.py", line 112, in forward
x = blk(x)
^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\segment_anything\modeling\image_encoder.py", line 174, in forward
x = self.attn(x)
^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\segment_anything\modeling\image_encoder.py", line 234, in forward
attn = add_decomposed_rel_pos(attn, q, self.rel_pos_h, self.rel_pos_w, (H, W), (H, W))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ImageAI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\segment_anything\modeling\image_encoder.py", line 358, in add_decomposed_rel_pos
attn.view(B, q_h, q_w, k_h, k_w) + rel_h[:, :, :, :, None] + rel_w[:, :, :, None, :]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
torch.OutOfMemoryError: Allocation on device

2024-09-07 03:23:05,525 - root - ERROR - Got an OOM, unloading all loaded models.
2024-09-07 03:23:05,654 - root - INFO - Prompt executed in 1235.57 seconds
2024-09-07 03:23:05,998 - root - INFO - Requested to load SDXL
2024-09-07 03:23:05,999 - root - INFO - Loading 1 new model
```
## Attached Workflow
Please make sure that workflow does not contain any sensitive information such as API keys or passwords.
```
Workflow too large. Please manually upload the workflow from local file system.
```

## Additional Context
(Please add any additional context or steps to reproduce the error here)
```

### Other

_No response_