Bug Report: ACE-Step 1.5 Integration Memory Leak / CPU Fallback causing severe slowdowns and OOM on Windows/AMD
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
### Custom Node Testing
- [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)
### Expected Behavior
Following recent repository updates, running the ACE-Step 1.5 workflow (using `ACE15TEModel_` and `AudioOobleckVAE`) under Windows 11 with AMD GPUs leads to severe generation slowdowns and Out-of-Memory (OOM) errors.
The issue appears to stem from a regression where the Language Model sampling step (`LM sampling`) silently drops the heavy text encoder loop back onto system RAM/CPU or fails to free the KV-cache between generations, completely bypassing available VRAM allocations. This behavior is consistently observable across multiple driver versions
### Actual Behavior
Following recent repository updates, running the ACE-Step 1.5 workflow (using `ACE15TEModel_` and `AudioOobleckVAE`) under Windows 11 with AMD GPUs leads to severe generation slowdowns and Out-of-Memory (OOM) errors.
The issue appears to stem from a regression where the Language Model sampling step (`LM sampling`) silently drops the heavy text encoder loop back onto system RAM/CPU or fails to free the KV-cache between generations, completely bypassing a
### Steps to Reproduce
1. Load a standard ACE-Step 1.5 music generation workflow in ComfyUI.
2. Load `ACE15TEModel_` (~9.1 GB) and `AudioOobleckVAE`.
3. Execute a prompt generation.
4. Observe the console output during the `LM sampling` phase.
### Debug Logs
```powershell
### Console Logs
[ERROR] Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle:
Traceback (most recent call last):
File "asyncio\events.py", line 88, in _run
File "asyncio\proactor_events.py", line 165, in _call_connection_lost
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
[INFO] got prompt
[INFO] VAE load device: cuda:0, offload device: cpu, dtype: torch.float16
[INFO] Requested to load ACE15TEModel_
[INFO] loaded completely; 9127.29 MB loaded, full load: True
[INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cuda:0, dtype: torch.float16
C:\ComfyUI_windows_portable_amd\ComfyUI_windows_portable\ComfyUI\comfy\ops.py:95: UserWarning: Using AOTriton backend for Efficient Attention forward... (Triggered internally at C:/b/pytorch/aten/src/ATen/native/transformers/hip/attention.hip:1452.)
return torch.nn.functional.scaled_dot_product_attention(q, k, v, *args, **kwargs)
LM sampling: 100%|███████████████████████████████████████████████████████████████████| 600/600 [00:22<00:00, 27.18it/s]
[INFO] model weight dtype torch.bfloat16, manual cast: None
[INFO] model_type FLOW
[INFO] Requested to load ACEStep15
[INFO] loaded completely; 5085.47 MB usable, 4565.35 MB loaded, full load: True
100%|████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:03<00:00, 2.10it/s]
[INFO] Requested to load AudioOobleckVAE
[INFO] Unloaded partially: 6440.50 MB freed, 2686.79 MB remains loaded, 237.50 MB buffer reserved, lowvram patches: 0
[INFO] loaded completely; 321.70 MB loaded, full load: True
[INFO] Prompt executed in 63.24 seconds
```
### Other
_No response_
Contributor guide
Research direction
Start with the ACE-Step 1.5 workflow and the console output around the `LM sampling` phase; inspect `comfy/ops.py` at line 95 and the model-loading messages for device and offload behavior. Reproduce the workflow on Windows with an AMD GPU, then verify that repeated generation does not fall back to CPU/system RAM, retain the KV-cache unexpectedly, or produce OOM errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100