aimdo:VRAM Allocation failed (non OOM) in 17.1
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 155
Description
### Custom Node Testing
- [ ] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)
### Expected Behavior
The attached example usually runs through completely and creates a video of someone constantly turning in circles.
### Actual Behavior
Sometimes it fails with "aimdo: src/model-vbar.c:307:ERROR:VRAM Allocation failed (non OOM)".
This mainly happens if image is scaled to a height of 720. For heights of 480 or 900 it always works. Clearly different image sizes use different amounts of VRAM for the output latent so the juggling that aimdo does with the models changes.
### Steps to Reproduce
It is random whether it runs or not. Fortunately I managed to capture a run with --verbose DEBUG which gives a hint to the failure mechanism:
aimdo: D:\a\comfy-aimdo\comfy-aimdo\src\plat.h:142:DEBUG:CUDA API FAILED : err = cuMemSetAccess(vaddr, size, &accessDesc, 1) : device not ready
aimdo: src/model-vbar.c:307:ERROR:VRAM Allocation failed (non OOM)
So it looks like a race condition in the third of the "three stooges" in model-vbar.c
I have attached a log of a failing run (no verbose output) with two fails in full and the last part of a run with --verbose DEBUG.
(Unfortunately I don't have the whole of the DEBUG run log.)
The workflow which causes the error. It can occur at any of the occasions where the WAN21 model is loaded.
[causeaimdoerror.json](https://github.com/user-attachments/files/25995909/causeaimdoerror.json)
The example jpg to use with the workflow. It does seem to matter how big everything is with respect to the size of VRAM. (I have 16GB).
Log of two failing runs
[failingrunexamplelog_nodebuginfo.txt](https://github.com/user-attachments/files/25995880/failingrunexamplelog_nodebuginfo.txt)
Verbose debug log
[partialfailingrun_verbosedebug.txt](https://github.com/user-attachments/files/25995883/partialfailingrun_verbosedebug.txt)
### Debug Logs
```powershell
Microsoft Windows [Version 10.0.26200.8037]
(c) Microsoft Corporation. All rights reserved.
M:\pinokio\api\comfy.git\app>conda_hook & conda deactivate & conda deactivate & conda deactivate & conda activate base & M:\pinokio\api\comfy.git\app\env\Scripts\activate M:\pinokio\api\comfy.git\app\env && python main.py --cache-none --use-sage-attention
[START] Security scan
[ComfyUI-Manager] `python -m pip` not available. Falling back to `uv`.
[ComfyUI-Manager] Using `uv` as Python module for pip operations.
Using Python 3.13.11 environment at: M:\pinokio\api\experiment\app\env
[DONE] Security scan
## ComfyUI-Manager: installing dependencies done.
** ComfyUI startup time: 2026-03-14 12:05:51.429
** Platform: Windows
** Python version: 3.13.11 (main, Dec 17 2025, 21:09:15) [MSC v.1944 64 bit (AMD64)]
** Python executable: M:\pinokio\api\experiment\app\env\Scripts\python.exe
** ComfyUI Path: M:\pinokio\api\comfy.git\app
** ComfyUI Base Folder Path: M:\pinokio\api\comfy.git\app
** User directory: M:\pinokio\api\comfy.git\app\user
** ComfyUI-Manager config path: M:\pinokio\api\comfy.git\app\user\__manager\config.ini
** Log path: M:\pinokio\api\comfy.git\app\user\comfyui.log
Using Python 3.13.11 environment at: M:\pinokio\api\experiment\app\env
Using Python 3.13.11 environment at: M:\pinokio\api\experiment\app\env
Prestartup times for custom nodes:
0.0 seconds: M:\pinokio\api\comfy.git\app\custom_nodes\comfyui-easy-use
0.6 seconds: M:\pinokio\api\comfy.git\app\custom_nodes\ComfyUI-Manager
Found comfy_kitchen backend eager: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_mxfp8', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'scaled_mm_mxfp8', 'scaled_mm_nvfp4']}
Found comfy_kitchen backend triton: {'available': True, 'disabled': True, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8']}
Found comfy_kitchen backend cuda: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'scaled_mm_nvfp4']}
Checkpoint files will always be loaded safely.
Total VRAM 16376 MB, total RAM 65129 MB
pytorch version: 2.10.0+cu130
xformers version: 0.0.35
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4070 Ti SUPER : cudaMallocAsync
Using async weight offloading with 2 streams
Enabled pinned memory 29308.0
working around nvidia conv3d memory bug.
Using sage attention
aimdo: src-win/cuda-detour.c:77:INFO:aimdo_setup_hooks: found driver at 00007FFAFCBF0000, installing 4 hooks
aimdo: src-win/cuda-detour.c:61:DEBUG:install_hook_entrys: hooks successfully installed
aimdo: src/control.c:69:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce RTX 4070 Ti SUPER (VRAM: 16375 MB)
DynamicVRAM support detected and enabled
Python version: 3.13.11 (main, Dec 17 2025, 21:09:15) [MSC v.1944 64 bit (AMD64)]
ComfyUI version: 0.17.0
comfy-aimdo version: 0.2.11
comfy-kitchen version: 0.2.8
ComfyUI frontend version: 1.41.19
[Prompt Server] web root: M:\pinokio\api\experiment\app\env\Lib\site-packages\comfyui_frontend_package\static
Asset seeder disabled
[ComfyUI-Easy-Use] server: v1.3.4 Loaded
[ComfyUI-Easy-Use] web root: M:\pinokio\api\comfy.git\app\custom_nodes\comfyui-easy-use\web_version/v2 Loaded
ComfyUI-GGUF: Allowing full torch compile
### Loading: ComfyUI-Manager (V3.39)
[ComfyUI-Manager] network_mode: public
[ComfyUI-Manager] ComfyUI per-queue preview override detected (PR #11261). Manager's preview method feature is disabled. Use ComfyUI's --preview-method CLI option or 'Settings > Execution > Live preview method'.
### ComfyUI Version: v0.17.0-3-g6cd35a0c | Released on '2026-03-13'
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
'sox' is not recognized as an internal or external command,
operable program or batch file.
SoX could not be found!
If you do not have SoX, proceed here:
- - - http://sox.sourceforge.net/ - - -
If you do (or think that you should) have SoX, double-check your
path variables.
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
✅ ComfyUI-Qwen-TTS v1.0.6 loaded
Import times for custom nodes:
0.0 seconds: M:\pinokio\api\comfy.git\app\custom_nodes\websocket_image_save.py
0.0 seconds: M:\pinokio\api\comfy.git\app\custom_nodes\ComfyUI_JPS-Nodes
0.0 seconds: M:\pinokio\api\comfy.git\app\custom_nodes\ComfyUI-GGUF
0.0 seconds: M:\pinokio\api\comfy.git\app\custom_nodes\ComfyUI-Notifications
0.0 seconds: M:\pinokio\api\comfy.git\app\custom_nodes\comfyui-kjnodes
0.0 seconds: M:\pinokio\api\comfy.git\app\custom_nodes\ControlFlowUtils
0.1 seconds: M:\pinokio\api\comfy.git\app\custom_nodes\comfyui-videohelpersuite
0.2 seconds: M:\pinokio\api\comfy.git\app\custom_nodes\comfyui-easy-dotsocr
0.6 seconds: M:\pinokio\api\comfy.git\app\custom_nodes\ComfyUI-Manager
0.7 seconds: M:\pinokio\api\comfy.git\app\custom_nodes\qwen3-tts-comfyui
2.3 seconds: M:\pinokio\api\comfy.git\app\custom_nodes\comfyui-easy-use
Context impl SQLiteImpl.
Will assume non-transactional DDL.
Disabling intermediate node cache.
Starting server
To see the GUI go to: http://127.0.0.1:8188
[DEPRECATION WARNING] Detected import of deprecated legacy API: /scripts/ui.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
[DEPRECATION WARNING] Detected import of deprecated legacy API: /extensions/core/groupNode.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
[DEPRECATION WARNING] Detected import of deprecated legacy API: /extensions/core/widgetInputs.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
[DEPRECATION WARNING] Detected import of deprecated legacy API: /scripts/ui/components/buttonGroup.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
[DEPRECATION WARNING] Detected import of deprecated legacy API: /scripts/ui/components/button.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
Exception in callback _ProactorBasePipeTransport._call_connection_lost()
handle:
Traceback (most recent call last):
File "C:\Users\njcr\AppData\Roaming\uv\python\cpython-3.13.11-windows-x86_64-none\Lib\asyncio\events.py", line 89, in _run
self._context.run(self._callback, *self._args)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\njcr\AppData\Roaming\uv\python\cpython-3.13.11-windows-x86_64-none\Lib\asyncio\proactor_events.py", line 165, in _call_connection_lost
self._sock.shutdown(socket.SHUT_RDWR)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
FETCH ComfyRegistry Data: 5/130
FETCH ComfyRegistry Data: 10/130
FETCH ComfyRegistry Data: 15/130
FETCH ComfyRegistry Data: 20/130
FETCH ComfyRegistry Data: 25/130
FETCH ComfyRegistry Data: 30/130
FETCH ComfyRegistry Data: 35/130
FETCH ComfyRegistry Data: 40/130
FETCH ComfyRegistry Data: 45/130
FETCH ComfyRegistry Data: 50/130
FETCH ComfyRegistry Data: 55/130
FETCH ComfyRegistry Data: 60/130
FETCH ComfyRegistry Data: 65/130
FETCH ComfyRegistry Data: 70/130
FETCH ComfyRegistry Data: 75/130
FETCH ComfyRegistry Data: 80/130
FETCH ComfyRegistry Data: 85/130
FETCH ComfyRegistry Data: 90/130
FETCH ComfyRegistry Data: 95/130
FETCH ComfyRegistry Data: 100/130
FETCH ComfyRegistry Data: 105/130
FETCH ComfyRegistry Data: 110/130
FETCH ComfyRegistry Data: 115/130
FETCH ComfyRegistry Data: 120/130
FETCH ComfyRegistry Data: 125/130
FETCH ComfyRegistry Data: 130/130
FETCH ComfyRegistry Data [DONE]
[ComfyUI-Manager] default cache updated: https://api.comfy.org/nodes
FETCH DATA from: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json [DONE]
[ComfyUI-Manager] All startup tasks have been completed.
got prompt
Using xformers attention in VAE
Using xformers attention in VAE
VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
Requested to load WanVAE
Model WanVAE prepared for dynamic VRAM loading. 242MB Staged. 0 patches attached. Force pre-loaded 52 weights: 28 KB.
Found quantization metadata version 1
Using MixedPrecisionOps for text encoder
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load WanTEModel
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
Found quantization metadata version 1
Detected mixed precision quantization
Using mixed precision operations
model weight dtype torch.float16, manual cast: torch.float16
model_type FLOW
Using sage attention mode: auto
Requested to load WAN21
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:32<00:00, 10.80s/it]
Found quantization metadata version 1
Detected mixed precision quantization
Using mixed precision operations
model weight dtype torch.float16, manual cast: torch.float16
model_type FLOW
Using sage attention mode: auto
Requested to load WAN21
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:32<00:00, 10.82s/it]
Model WanVAE prepared for dynamic VRAM loading. 242MB Staged. 0 patches attached. Force pre-loaded 52 weights: 28 KB.
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
0 models unloaded.
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
0 models unloaded.
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:36<00:00, 12.22s/it]
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:36<00:00, 12.24s/it]
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
0 models unloaded.
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:32<00:00, 10.77s/it]
0 models unloaded.
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:32<00:00, 10.85s/it]
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
0 models unloaded.
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:36<00:00, 12.30s/it]
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:36<00:00, 12.25s/it]
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
0 models unloaded.
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:34<00:00, 11.56s/it]
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:34<00:00, 11.44s/it]
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
0 models unloaded.
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:34<00:00, 11.53s/it]
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:34<00:00, 11.34s/it]
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
0 models unloaded.
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
0 models unloaded.
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:34<00:00, 11.59s/it]
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:34<00:00, 11.40s/it]
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
0 models unloaded.
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
0 models unloaded.
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:34<00:00, 11.55s/it]
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:34<00:00, 11.41s/it]
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
0 models unloaded.
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 49 weights: 392 KB.
0 models unloaded.
Model WAN21 prepared for dynamic VRAM loading. 13631MB Staged. 400 patches attached.
0%| | 0/3 [00:00...
comfy.patcher_extension.get_all_wrappers(comfy.patcher_extension.WrappersMP.APPLY_MODEL, transformer_options)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
).execute(x, t, c_concat, c_crossattn, control, transformer_options, **kwargs)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "M:\pinokio\api\comfy.git\app\comfy\patcher_extension.py", line 112, in execute
return self.original(*args, **kwargs)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "M:\pinokio\api\comfy.git\app\comfy\model_base.py", line 209, in _apply_model
model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds)
File "M:\pinokio\api\experiment\app\env\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "M:\pinokio\api\experiment\app\env\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
return forward_call(*args, **kwargs)
File "M:\pinokio\api\comfy.git\app\comfy\ldm\wan\model.py", line 644, in forward
return comfy.patcher_extension.WrapperExecutor.new_class_executor(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...<2 lines>...
comfy.patcher_extension.get_all_wrappers(comfy.patcher_extension.WrappersMP.DIFFUSION_MODEL, transformer_options)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
).execute(x, timestep, context, clip_fea, time_dim_concat, transformer_options, **kwargs)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "M:\pinokio\api\comfy.git\app\comfy\patcher_extension.py", line 112, in execute
return self.original(*args, **kwargs)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "M:\pinokio\api\comfy.git\app\comfy\ldm\wan\model.py", line 664, in _forward
return self.forward_orig(x, timestep, context, clip_fea=clip_fea, freqs=freqs, transformer_options=transformer_options, **kwargs)[:, :, :t, :h, :w]
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "M:\pinokio\api\comfy.git\app\comfy\ldm\wan\model.py", line 597, in forward_orig
x = block(x, e=e0, freqs=freqs, context=context, context_img_len=context_img_len, transformer_options=transformer_options)
File "M:\pinokio\api\experiment\app\env\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "M:\pinokio\api\experiment\app\env\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
return forward_call(*args, **kwargs)
File "M:\pinokio\api\comfy.git\app\comfy\ldm\wan\model.py", line 258, in forward
y = self.ffn(torch.addcmul(repeat_e(e[3], x), self.norm2(x), 1 + repeat_e(e[4], x)))
File "M:\pinokio\api\experiment\app\env\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "M:\pinokio\api\experiment\app\env\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
return forward_call(*args, **kwargs)
File "M:\pinokio\api\experiment\app\env\Lib\site-packages\torch\nn\modules\container.py", line 253, in forward
input = module(input)
File "M:\pinokio\api\experiment\app\env\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "M:\pinokio\api\experiment\app\env\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
return forward_call(*args, **kwargs)
File "M:\pinokio\api\comfy.git\app\comfy\ops.py", line 908, in forward
output = self.forward_comfy_cast_weights(input, compute_dtype, want_requant=isinstance(input, QuantizedTensor))
File "M:\pinokio\api\comfy.git\app\comfy\ops.py", line 878, in forward_comfy_cast_weights
weight, bias, offload_stream = cast_bias_weight(self, input, offloadable=True, compute_dtype=compute_dtype, want_requant=want_requant)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "M:\pinokio\api\comfy.git\app\comfy\ops.py", line 228, in cast_bias_weight
return cast_bias_weight_with_vbar(s, dtype, device, bias_dtype, non_blocking, compute_dtype, want_requant)
File "M:\pinokio\api\comfy.git\app\comfy\ops.py", line 101, in cast_bias_weight_with_vbar
signature = comfy_aimdo.model_vbar.vbar_fault(s._v)
File "M:\pinokio\api\experiment\app\env\Lib\site-packages\comfy_aimdo\model_vbar.py", line 127, in vbar_fault
return vbar.fault(offset, size)
~~~~~~~~~~^^^^^^^^^^^^^^
File "M:\pinokio\api\experiment\app\env\Lib\site-packages\comfy_aimdo\model_vbar.py", line 89, in fault
raise RuntimeError(f"Fault failed: {res}")
RuntimeError: Fault failed: 2
Prompt executed in 00:10:22
```
### Other
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.