Comfy-Org / Comfy-Org/ComfyUI

easycache + torch compile doesnt work

Open
#14,795 0 comments 0 reactions 0 assignees View on GitHub
Potential Bug
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Custom Node Testing

- [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Expected Behavior

for the image to gen at reasonable speeds

### Actual Behavior

after comfyui boots up, if the first image is generated with both easycache and torchcompile, it is very very very slow and it gives a bunch of errors, but it does eventually generate.

if the first image is generated with torch compile and easycache is added after, easycache doesnt do anything. no steps are skipped and results are identical to without easycache.

### Steps to Reproduce

example workflow
[anima.json](https://github.com/user-attachments/files/29710692/anima.json)

### Debug Logs

```powershell
this is the log for generating the first image after comfy boots with torch compile and easycache

[INFO] setup plugin alembic.autogenerate.schemas
[INFO] setup plugin alembic.autogenerate.tables
[INFO] setup plugin alembic.autogenerate.types
[INFO] setup plugin alembic.autogenerate.constraints
[INFO] setup plugin alembic.autogenerate.defaults
[INFO] setup plugin alembic.autogenerate.comments
[INFO] Adding extra search path checkpoints I:\aistuff\models\checkpoints
[INFO] Adding extra search path clip I:\aistuff\models\clip
[INFO] Adding extra search path clip_vision I:\aistuff\models\clip_vision
[INFO] Adding extra search path configs I:\aistuff\models\configs
[INFO] Adding extra search path controlnet I:\aistuff\models\controlnet
[INFO] Adding extra search path diffusion_models I:\aistuff\models\diffusion_models
[INFO] Adding extra search path diffusion_models I:\aistuff\models\unet
[INFO] Adding extra search path embeddings I:\aistuff\models\embeddings
[INFO] Adding extra search path loras I:\aistuff\models\loras
[INFO] Adding extra search path upscale_models I:\aistuff\models\upscale_models
[INFO] Adding extra search path vae I:\aistuff\models\vae
[INFO] Adding extra search path text_encoders I:\aistuff\models\text_encoders
[INFO] Adding extra search path inpaint I:\aistuff\models\inpaint
[INFO] Adding extra search path model_patches I:\aistuff\models\model_patches
[START] Security scan
[INFO] [ComfyUI-Manager] Using uv as Python module for pip operations.
Using Python 3.13.14 environment at: venv
[DONE] Security scan
** ComfyUI startup time: 2026-07-06 19:24:48.836
** Platform: Windows
** Python version: 3.13.14 (tags/v3.13.14:fd17997, Jun 10 2026, 13:03:48) [MSC v.1944 64 bit (AMD64)]
** Python executable: I:\aistuff\ComfyUI\venv\Scripts\python.exe
** ComfyUI Path: I:\aistuff\ComfyUI
** ComfyUI Base Folder Path: I:\aistuff\ComfyUI
** User directory: I:\aistuff\ComfyUI\user
** ComfyUI-Manager config path: I:\aistuff\ComfyUI\user\__manager\config.ini
** Log path: I:\aistuff\ComfyUI\user\comfyui.log
Using Python 3.13.14 environment at: venv
Using Python 3.13.14 environment at: venv
[INFO] [PRE] ComfyUI-Manager
[INFO] Found comfy_kitchen backend eager: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['adaln', 'apply_rope', 'apply_rope1', 'apply_rope_split_half', 'apply_rope_split_half1', 'dequantize_int8_convrot_weight', 'dequantize_int8_convrot_weight_dtype', 'dequantize_int8_simple', 'dequantize_int8_simple_dtype', 'dequantize_mxfp8', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'gemv_awq_w4a16', 'int8_linear', 'quantize_and_rotate_rowwise', 'quantize_int8_convrot_weight', 'quantize_int8_rowwise', 'quantize_int8_tensorwise', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'quantize_svdquant_w4a4', 'scaled_mm_mxfp8', 'scaled_mm_nvfp4', 'scaled_mm_svdquant_w4a4', 'stochastic_rounding_fp8']}
[INFO] Found comfy_kitchen backend cuda: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['adaln', 'apply_rope', 'apply_rope1', 'apply_rope_split_half', 'apply_rope_split_half1', 'dequantize_int8_convrot_weight', 'dequantize_int8_convrot_weight_dtype', 'dequantize_int8_simple', 'dequantize_int8_simple_dtype', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'gemv_awq_w4a16', 'int8_linear', 'quantize_and_rotate_rowwise', 'quantize_int8_convrot_weight', 'quantize_int8_rowwise', 'quantize_int8_tensorwise', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'quantize_svdquant_w4a4', 'scaled_mm_nvfp4', 'scaled_mm_svdquant_w4a4', 'stochastic_rounding_fp8']}
[INFO] Found comfy_kitchen backend triton: {'available': True, 'disabled': True, 'unavailable_reason': None, 'capabilities': ['adaln', 'apply_rope', 'apply_rope1', 'apply_rope_split_half', 'apply_rope_split_half1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'int8_linear', 'quantize_and_rotate_rowwise', 'quantize_int8_rowwise', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8']}
[INFO] Checkpoint files will always be loaded safely.
[INFO] Total VRAM 24564 MB, total RAM 31902 MB
[INFO] pytorch version: 2.12.1+cu132
[INFO] Set vram state to: NORMAL_VRAM
[INFO] Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
[INFO] Using async weight offloading with 2 streams
[INFO] Enabled pinned memory 12760.0
[INFO] Using pytorch attention
aimdo: src-win/cuda-detour.c:38:INFO:aimdo_setup_hooks: installing 6 hooks
aimdo: src-win/cuda-detour.c:28:DEBUG:install_hook_entries: hooks successfully installed
aimdo: src-win/shmem-detect.c:80:INFO:comfy-aimdo WDDM adapter match: NVIDIA GeForce RTX 4090 runtime_luid=00000000:00012346 dxgi_luid=00000000:00012346
aimdo: src/control.c:248:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce RTX 4090 (VRAM: 24563 MB)
[INFO] DynamicVRAM support detected and enabled
[INFO] Python version: 3.13.14 (tags/v3.13.14:fd17997, Jun 10 2026, 13:03:48) [MSC v.1944 64 bit (AMD64)]
[INFO] ComfyUI version: 0.27.0
[INFO] comfy-aimdo version: 0.4.10
[INFO] comfy-kitchen version: 0.2.16
[INFO] comfyui-frontend-package version: 1.45.20
[INFO] comfyui-workflow-templates version: 0.11.2
[INFO] comfyui-embedded-docs version: 0.5.7
[INFO] comfy-kitchen version: 0.2.16
[INFO] comfy-aimdo version: 0.4.10
[INFO] [Prompt Server] web root: I:\aistuff\ComfyUI\venv\Lib\site-packages\comfyui_frontend_package\static
[INFO] Asset seeder disabled
[INFO] [START] ComfyUI-Manager
[ComfyUI-Manager] Using GitPython backend
[INFO] [ComfyUI-Manager] network_mode: public
[WARNING] [ComfyUI-Manager] The matrix sharing feature has been disabled because the `matrix-nio` dependency is not installed.
To use this feature, please run the following command:
I:\aistuff\ComfyUI\venv\Scripts\python.exe -m pip install matrix-nio

[INFO] No OpenGL_accelerate module loaded: No module named 'OpenGL_accelerate'
(Booru Tagger:BooruTagger) [DEBUG] Available ORT providers: AzureExecutionProvider, CPUExecutionProvider
(Booru Tagger:BooruTagger) [DEBUG] Using ORT providers: CPUExecutionProvider
[INFO]
Import times for custom nodes:
[INFO] 0.0 seconds: I:\aistuff\ComfyUI\custom_nodes\websocket_image_save.py
[INFO] 0.0 seconds: I:\aistuff\ComfyUI\custom_nodes\ComfyUI-ppm
[INFO] 0.6 seconds: I:\aistuff\ComfyUI\custom_nodes\ComfyUI-Booru-Tagger
[INFO]
[INFO] Context impl SQLiteImpl.
[INFO] Will assume non-transactional DDL.
[INFO] Using RAM pressure cache.
[INFO] Starting server

[INFO] To see the GUI go to: http://127.0.0.1:8188
[INFO] [ComfyUI-Manager] The ComfyRegistry cache update is still in progress, so an outdated cache is being used.
FETCH DATA from: I:\aistuff\ComfyUI\user\__manager\cache\1514988643_custom-node-list.json [DONE]
[INFO] got prompt
[INFO] Using pytorch attention in VAE
[INFO] Using pytorch attention in VAE
[INFO] VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
[INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
[INFO] Requested to load AnimaTEModel_
[INFO] Model AnimaTEModel_ prepared for dynamic VRAM loading. 1136MB Staged. 0 patches attached. Force pre-loaded 113 weights: 128 KB.
[INFO] Model AnimaTEModel_ prepared for dynamic VRAM loading. 1136MB Staged. 0 patches attached. Force pre-loaded 113 weights: 128 KB.
[INFO] model weight dtype torch.bfloat16, manual cast: None
[INFO] model_type FLOW
[INFO] model weight dtype torch.bfloat16, manual cast: None
[INFO] model_type FLOW
[INFO] EasyCache enabled - threshold: 0.05, start_percent: 0.15, end_percent: 0.95
[INFO] Requested to load Anima
[INFO] loaded completely; 20268.55 MB usable, 3988.40 MB loaded, full load: True
0%| | 0/30 [00:00 self.end_t).item()
W0706 19:26:20.524000 23752 venv\Lib\site-packages\torch\_dynamo\variables\tensor.py:1612] [0/0]
W0706 19:26:20.524000 23752 venv\Lib\site-packages\torch\_dynamo\variables\tensor.py:1612] [0/0]
23%|███████████████████▎ | 7/30 [00:55<13:07, 34.22s/it][INFO] FETCH ComfyRegistry Data [DONE]
[INFO] [ComfyUI-Manager] default cache updated: https://api.comfy.org/nodes
FETCH DATA from: I:\aistuff\ComfyUI\user\__manager\cache\1514988643_custom-node-list.json [DONE]
[INFO] [ComfyUI-Manager] All startup tasks have been completed.
I:\aistuff\ComfyUI\venv\Lib\site-packages\torch\_dynamo\variables\functions.py:2311: UserWarning: Dynamo does not know how to trace the builtin `.list.append.` This function is either a Python builtin (e.g. _warnings.warn) or a third-party C/C++ Python extension (perhaps created with pybind).
If it is a Python builtin, please file an issue on GitHub so the PyTorch team can add support for it and see the next case for a workaround.
If it is a third-party C/C++ Python extension, please either wrap it into a PyTorch-understood custom operator (see https://pytorch.org/tutorials/advanced/custom_ops_landing_page.html for more details) or, if it is traceable, use `torch.compiler.allow_in_graph`.
torch._dynamo.utils.warn_once(explanation + "\n" + "\n".join(hints))
67%|██████████████████████████████████████████████████████▋ | 20/30 [03:47<01:42, 10.21s/it]W0706 19:30:52.014000 23752 venv\Lib\site-packages\torch\_dynamo\convert_frame.py:1853] [15/8] torch._dynamo hit config.recompile_limit (8)
W0706 19:30:52.014000 23752 venv\Lib\site-packages\torch\_dynamo\convert_frame.py:1853] [15/8] function: 'torch_dynamo_resume_in_easycache_forward_wrapper_at_61' (I:\aistuff\ComfyUI\comfy_extras\nodes_easycache.py:61)
W0706 19:30:52.014000 23752 venv\Lib\site-packages\torch\_dynamo\convert_frame.py:1853] [15/8] last reason: 15/7: len(easycache.approx_output_change_rates) == 7 # easycache.approx_output_change_rates.append(approx_output_change_rate.item()) # comfy_extras\nodes_easycache.py:85 in torch_dynamo_resume_in_easycache_forward_wrapper_at_61
W0706 19:30:52.014000 23752 venv\Lib\site-packages\torch\_dynamo\convert_frame.py:1853] [15/8] User stack trace:
W0706 19:30:52.014000 23752 venv\Lib\site-packages\torch\_dynamo\convert_frame.py:1853] [15/8] File "I:\aistuff\ComfyUI\comfy_extras\nodes_easycache.py", line 85, in torch_dynamo_resume_in_easycache_forward_wrapper_at_61
W0706 19:30:52.014000 23752 venv\Lib\site-packages\torch\_dynamo\convert_frame.py:1853] [15/8] easycache.approx_output_change_rates.append(approx_output_change_rate.item())
W0706 19:30:52.014000 23752 venv\Lib\site-packages\torch\_dynamo\convert_frame.py:1853] [15/8] To log all recompilation reasons, use TORCH_LOGS="recompiles".
W0706 19:30:52.014000 23752 venv\Lib\site-packages\torch\_dynamo\convert_frame.py:1853] [15/8] To diagnose recompilation issues, see https://docs.pytorch.org/docs/main/user_guide/torch_compiler/compile/programming_model.recompilation.html
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [04:18<00:00, 8.62s/it]
[INFO] EasyCache - skipped 10/30 steps (1.50x speedup).
[INFO] Requested to load WanVAE
[INFO] Model WanVAE prepared for dynamic VRAM loading. 241MB Staged. 0 patches attached. Force pre-loaded 60 weights: 61 KB.
[INFO] Prompt executed in 306.69 seconds
```

### Other

torch compile + easycache used to be fine before, but im not sure what i changed for it to break

i also tried TORCHDYNAMO_CAPTURE_SCALAR_OUTPUTS=1 environment variable (i already have cl.exe in my path), it instead gave a compiler error.

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with the linked anima.json workflow, using the torch.compile and EasyCache combination described in the report. Start with comfy_extras/nodes_easycache.py, then follow the stack through comfy/ldm/anima/model.py, comfy/ldm/cosmos/predict2.py, and comfy/patcher_extension.py; done means the first generation is no longer excessively slow and EasyCache still skips steps when enabled after compilation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
backend, machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.