Comfy-Org / Comfy-Org/ComfyUI

ACE Step audio generation slowdown caused by VRAM eviction triggered by deprecated API hooks and conflicting custom nodes

Open
#15,974 1 comment 0 reactions 0 assignees View on GitHub
Potential Bug
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Custom Node Testing

- [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Expected Behavior

## Expected Behavior
ACE Step should maintain stable VRAM allocation throughout audio generation, without being affected by unrelated UI‑patching or multi‑model nodes.

### Actual Behavior

## Observed Symptoms

### 1. Severe slowdown in ACE Step audio generation
- 90‑second songs take **10+ minutes**
- 4‑minute songs take **20 minutes**

### 2. Repeated VRAM unloading and reloading during generation
Example log excerpts:
```
Unloaded partially: 5585.50 MB freed
Unloaded partially: 4177.50 MB freed
loaded partially; 8676.58 MB usable, 8651.29 MB loaded, 476.00 MB offloaded
```

### 3. Deprecated API warnings at startup
```
[DEPRECATION WARNING] Detected import of deprecated legacy API: /scripts/ui.js
[DEPRECATION WARNING] Detected import of deprecated legacy API: /extensions/core/clipspace.js
[DEPRECATION WARNING] Detected import of deprecated legacy API: /extensions/core/widgetInputs.js
```

### 4. Non‑deterministic caching warnings from rgthree-comfy
```
[rgthree-comfy][Power Puter] cannot be cached b/c it's using a non-deterministic function call
```

### 5. Connection reset errors during LM sampling
```
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
```

---

## Analysis
After isolating nodes and testing ACE Step in a reduced environment, the slowdown appears to be caused by **external custom nodes that patch ComfyUI’s UI or load additional heavy models**, resulting in:

- VRAM fragmentation
- forced model unload/reload cycles
- instability during ACE Step’s audio diffusion phase

Nodes that contributed to the issue include:

- `rgthree-comfy`
- `comfyui-workflow-prettier`
- `comfyui_fill-nodes`
- `comfyui-advanced-controlnet`
- `comfyui-animatediff-evolved`
- `comfyui-videohelpersuite`
- `comfyui-apex-artist`
- `comfyui-ollamagemini`

Disabling these nodes restored ACE Step’s normal performance and eliminated VRAM eviction during generation.

---

### Steps to Reproduce

## Analysis
After isolating nodes and testing ACE Step in a reduced environment, the slowdown appears to be caused by **external custom nodes that patch ComfyUI’s UI or load additional heavy models**, resulting in:

- VRAM fragmentation
- forced model unload/reload cycles
- instability during ACE Step’s audio diffusion phase

Nodes that contributed to the issue include:

- `rgthree-comfy`
- `comfyui-workflow-prettier`
- `comfyui_fill-nodes`
- `comfyui-advanced-controlnet`
- `comfyui-animatediff-evolved`
- `comfyui-videohelpersuite`
- `comfyui-apex-artist`
- `comfyui-ollamagemini`

Disabling these nodes restored ACE Step’s normal performance and eliminated VRAM eviction during generation.

### Debug Logs

```powershell
FETCH DATA from: C:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\comfyui_manager\custom-node-list.json [DONE]
[WARNING] [DEPRECATION WARNING] Detected import of deprecated legacy API: /scripts/ui.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
[WARNING] [DEPRECATION WARNING] Detected import of deprecated legacy API: /extensions/core/clipspace.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
[WARNING] [DEPRECATION WARNING] Detected import of deprecated legacy API: /extensions/core/widgetInputs.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
Error fetching Gemini models: 400 API key not valid. Please pass a valid API key. [reason: "API_KEY_INVALID"
domain: "googleapis.com"
metadata {
key: "service"
value: "generativelanguage.googleapis.com"
}
, locale: "en-US"
message: "API key not valid. Please pass a valid API key."
]
Using default model list.
Error fetching Gemini models: 400 API key not valid. Please pass a valid API key. [reason: "API_KEY_INVALID"
domain: "googleapis.com"
metadata {
key: "service"
value: "generativelanguage.googleapis.com"
}
, locale: "en-US"
message: "API key not valid. Please pass a valid API key."
]
Using default model list.
Error fetching Gemini image models: 400 API key not valid. Please pass a valid API key. [reason: "API_KEY_INVALID"
domain: "googleapis.com"
metadata {
key: "service"
value: "generativelanguage.googleapis.com"
}
, locale: "en-US"
message: "API key not valid. Please pass a valid API key."
]
Using default image model list.
Error fetching Gemini image models: 400 API key not valid. Please pass a valid API key. [reason: "API_KEY_INVALID"
domain: "googleapis.com"
metadata {
key: "service"
value: "generativelanguage.googleapis.com"
}
, locale: "en-US"
message: "API key not valid. Please pass a valid API key."
]
Using default image model list.
[WARNING] [DEPRECATION WARNING] Detected import of deprecated legacy API: /scripts/ui/components/button.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
[INFO] HTTP Request: GET http://127.0.0.1:11434/api/tags "HTTP/1.1 200 OK"
[INFO] HTTP Request: GET http://127.0.0.1:11434/api/tags "HTTP/1.1 200 OK"
[INFO] HTTP Request: GET http://127.0.0.1:11434/api/tags "HTTP/1.1 200 OK"
[INFO] got prompt
[rgthree-comfy][Power Puter] Note, Power Puter (node #189) cannot be cached b/c it's using a non-deterministic function call. Matches function call for `random.int`.
[rgthree-comfy][Power Puter] Note, Power Puter (node #190) cannot be cached b/c it's using a non-deterministic function call. Matches function call for `random.int`.
[rgthree-comfy][Power Puter] Power Puter node #190 should update to use the `random.int` built-in instead of `random_int`.
[rgthree-comfy][Power Puter] Power Puter node #189 should update to use the `random.int` built-in instead of `random_int`.
[INFO] VAE load device: cuda:0, offload device: cpu, dtype: torch.float16
[INFO] FETCH ComfyRegistry Data [DONE]
[INFO] [ComfyUI-Manager] default cache updated: https://api.comfy.org/nodes
FETCH DATA from: C:\ComfyUI_windows_portable\ComfyUI\user\__manager\cache\1514988643_custom-node-list.json [DONE]
[INFO] [ComfyUI-Manager] All startup tasks have been completed.
[INFO] HTTP Request: POST http://127.0.0.1:11434/api/generate "HTTP/1.1 200 OK"
[INFO] HTTP Request: POST http://127.0.0.1:11434/api/generate "HTTP/1.1 200 OK"
[INFO] HTTP Request: POST http://127.0.0.1:11434/api/generate "HTTP/1.1 200 OK"
[EasyUse] Save Text: Saving to ComfyUI\output\lyrics\Ritual Rust_postpunk.txt
[INFO] Requested to load ACE15TEModel_
[INFO] loaded completely; 9127.29 MB loaded, full load: True
[INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cuda:0, dtype: torch.float16
[WARNING] C:\ComfyUI_windows_portable\ComfyUI\comfy\ops.py:95: UserWarning: Using AOTriton backend for Efficient Attention forward... (Triggered internally at C:/b/pytorch/aten/src/ATen/native/transformers/hip/attention.hip:1452.)
return torch.nn.functional.scaled_dot_product_attention(q, k, v, *args, **kwargs)

LM sampling: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 470/470 [00:18<00:00, 25.48it/s]
[INFO] model weight dtype torch.bfloat16, manual cast: None
[INFO] model_type FLOW
[INFO] Requested to load ACEStep15
[INFO] Unloaded partially: 5585.50 MB freed, 3541.79 MB remains loaded, 237.50 MB buffer reserved, lowvram patches: 0
[INFO] loaded completely; 10758.75 MB usable, 9511.56 MB loaded, full load: True
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 42/42 [00:08<00:00, 4.74it/s]
[INFO] Requested to load AudioOobleckVAE
[INFO] Unloaded partially: 522.50 MB freed, 3019.29 MB remains loaded, 237.50 MB buffer reserved, lowvram patches: 0
[INFO] loaded completely; 321.70 MB loaded, full load: True
[INFO] Unloaded partially: 4177.50 MB freed, 5335.04 MB remains loaded, 72.00 MB buffer reserved, lowvram patches: 0
[INFO] loaded partially; 8676.58 MB usable, 8651.29 MB loaded, 476.00 MB offloaded, 25.00 MB buffer reserved, lowvram patches: 0
LM sampling: 77%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍ | 461/600 [00:27<00:11, 12.53it/s][ERROR] Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle:
Traceback (most recent call last):
File "asyncio\events.py", line 88, in _run
File "asyncio\proactor_events.py", line 165, in _call_connection_lost
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
LM sampling: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏| 597/600 [00:38<00:00, 13.61it/s]
```

### Other

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reproducing ACE Step generation with the listed custom nodes enabled and disabled, comparing the VRAM and deprecation logs. Read the referenced comfy/ops.py warning and deprecated entry points such as /scripts/ui.js and /extensions/core/clipspace.js; done requires a reproducible ComfyUI-owned cause rather than an issue isolated to external nodes.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, ollama, python, pytorch
Domain
ai, backend, frontend, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.