Comfy-Org / Comfy-Org/ComfyUI

--use-sage-attention flag causing CUDA illegal memory access

Open
#6,125 14 comments 0 reactions 0 assignees View on GitHub
Potential Bug
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Expected Behavior

Expected Hunyuan video output, but ComfyUI crashes instead.

### Actual Behavior

Hunyuan Wrapper from Kijai working fine, but for some reason sageattention isn't working correctly with comfyUI

### Steps to Reproduce

Runpod RTX4090 with python 3.11/cu124 template:
Install torch for cu124
Install sage attention 2.0.0
start ComfyUI using "python main.py --use-sage-attention"
Use the ComfyUI Example workflow
ComfyUI crashes at "SamplerCustomAdvanced" node
![Screenshot 2024-12-19 at 6 06 48 PM](https://github.com/user-attachments/assets/a01fd01b-41a0-4656-8b3c-18f7bdc1ca1e)

### Debug Logs

```powershell
/workspace/ComfyUI/sageattention/sageattention/core.py:631: UserWarning: pv_accum_dtype is 'fp32+fp32', smooth_v will be ignored.
warnings.warn("pv_accum_dtype is 'fp32+fp32', smooth_v will be ignored.")
terminate called after throwing an instance of 'c10::Error'
what(): CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Exception raised from c10_cuda_check_implementation at ../c10/cuda/CUDAException.cpp:43 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x96 (0x77fefe0b9446 in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::string const&) + 0x64 (0x77fefe0636e4 in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libc10.so)
frame #2: c10::cuda::c10_cuda_check_implementation(int, char const*, char const*, int, bool) + 0x118 (0x77fefe1a5a18 in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libc10_cuda.so)
frame #3: + 0x5e460 (0x77fefe1ab460 in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libc10_cuda.so)
frame #4: + 0x60184 (0x77fefe1ad184 in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libc10_cuda.so)
frame #5: + 0x3847bdc (0x77feb6247bdc in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libtorch_cuda.so)
frame #6: + 0x389a4c3 (0x77feb629a4c3 in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libtorch_cuda.so)
frame #7: + 0x389ced4 (0x77feb629ced4 in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libtorch_cuda.so)
frame #8: + 0x35e696c (0x77feb5fe696c in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libtorch_cuda.so)
frame #9: + 0x35e6a3d (0x77feb5fe6a3d in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libtorch_cuda.so)
frame #10: at::_ops::addmm::call(at::Tensor const&, at::Tensor const&, at::Tensor const&, c10::Scalar const&, c10::Scalar const&) + 0x19e (0x77fee9f65efe in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libtorch_cpu.so)
frame #11: + 0x1d46803 (0x77fee9746803 in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libtorch_cpu.so)
frame #12: at::native::linear(at::Tensor const&, at::Tensor const&, std::optional const&) + 0x6d3 (0x77fee97476a3 in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libtorch_cpu.so)
frame #13: + 0x2fed243 (0x77feea9ed243 in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libtorch_cpu.so)
frame #14: at::_ops::linear::call(at::Tensor const&, at::Tensor const&, std::optional const&) + 0x18c (0x77fee9f4e05c in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libtorch_cpu.so)
frame #15: + 0x75d5ac (0x77fefcf5d5ac in /workspace/ComfyUI/venv/lib/python3.11/site-packages/torch/lib/libtorch_python.so)
frame #16: python() [0x55563b]

frame #19: python() [0x5851f0]
frame #21: python() [0x5851f0]
frame #23: python() [0x5851f0]
frame #28: python() [0x659fdb]
frame #31: python() [0x585af7]
frame #32: python() [0x5852de]
frame #35: python() [0x585af7]
frame #36: python() [0x5852de]
frame #42: python() [0x659fdb]
frame #45: python() [0x585af7]
frame #46: python() [0x5852de]
frame #49: python() [0x585af7]
frame #50: python() [0x5852de]
frame #56: python() [0x659fdb]
frame #59: python() [0x585af7]
frame #60: python() [0x5852de]

Aborted (core dumped)
```

### Other

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.