Crash: CUDA error (C:/a/xformers/xformers/third_party/flash-attention/hopper\flash_fwd_launch_template.h:188): invalid argument`
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
### Custom Node Testing
- [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)
### Expected Behavior
Image is generated. If a crash does occur, even a fatal error, it should be caught, and guidance given to user to hopefully resolve issue.
### Actual Behavior
ComfyUI crashes a few seconds after run. It was working previously. I have been installing new custom nodes, so I assume the problem is related to a trashed environment. But this is a very new install, and it's frustrating that Comfy crashes without giving any guidance.
### Steps to Reproduce
1. install comfy
2. mess up your env as with below custom nodes and pip freeze report (I know, hard to test, and I have not tried to manually repro this)
3. disable extensions
4. run any basic image gen template workflow
5. crash
nodes folder
```
ComfyUI_Comfyroll_CustomNodes
comfyui_dagthomas
ComfyUI_PromptManager
ComfyUI_UltimateSDUpscale
ComfyUI-Copilot
ComfyUI-Custom-Scripts
comfyui-easy-use
ComfyUI-GGUF
ComfyUI-Griptape
comfyui-impact-pack
comfyui-inspire-pack
ComfyUI-joycaption-beta-one-GGUF
comfyui-kjnodes
ComfyUI-Lora-Manager
ComfyUI-Manager
comfyui-mape-helpers
ComfyUI-Upscaler-Tensorrt
ComfyUI-VideoHelperSuite
confyui-node-prompt-stash-saver
OneButtonPrompt
rgthree-comfy
was-node-suite-comfyui
wlsh_nodes
```
pip freeze:
```
absl-py==2.3.1
accelerate==1.9.0
addict==2.4.0
aiofiles==24.1.0
aiohappyeyeballs==2.6.1
aiohttp==3.12.15
aiohttp-socks==0.10.1
aiolimiter==1.2.1
aiosignal==1.4.0
albucore==0.0.24
albumentations==2.0.8
alembic==1.16.4
annotated-types==0.7.0
anthropic==0.59.0
antlr4-python3-runtime==4.9.3
anyio==4.9.0
asn1crypto==1.5.1
astrapy==2.0.1
async-timeout==5.0.1
attrs==25.3.0
audioread==3.0.1
authlib==1.6.1
av==15.0.0
babel==2.17.0
basicsr==1.4.2
beautifulsoup4==4.13.4
bitsandbytes==0.46.1
boto3==1.39.14
botocore==1.39.14
build==1.2.2.post1
cachecontrol==0.14.3
cachetools==5.5.2
certifi==2025.7.14
cffi==1.17.1
chardet==5.2.0
charset-normalizer==3.4.2
cleo==2.1.0
click==8.2.1
clip-interrogator==0.6.0
cmake==4.0.3
cohere==5.16.1
color-matcher==0.6.0
colorama==0.4.6
coloredlogs==15.0.1
comfyui-embedded-docs==0.2.4
comfyui-frontend-package==1.23.4
comfyui-workflow-templates==0.1.44
contourpy==1.3.2
courlan==1.3.2
crashtest==0.4.1
cryptography==45.0.5
cstr @ git+https://github.com/ltdrdata/cstr@0520c29a18a7a869a6e5983861d6f7a4c86f8e9b
cycler==0.12.1
cyclopts==3.22.3
cython==3.1.2
dateparser==1.2.2
ddt==1.7.2
decorator==5.2.1
deprecation==2.1.0
diffusers==0.34.0
dill==0.4.0
diskcache==5.6.3
distlib==0.4.0
distro==1.9.0
dnspython==2.7.0
docstring-parser==0.17.0
docutils==0.21.2
duckduckgo-search==8.1.1
dulwich==0.21.7
easydict==1.13
einops==0.8.1
elevenlabs==2.8.1
email-validator==2.2.0
exa-py==1.14.18
exceptiongroup==1.3.0
face-alignment==1.4.1
facexlib==0.3.0
fairscale==0.4.13
fastavro==1.11.1
fastjsonschema==2.21.1
fastmcp==2.10.6
ffmpy @ git+https://github.com/ltdrdata/ffmpy.git@f000737698b387ffaeab7cd871b0e9185811230d
filelock==3.18.0
filetype==1.2.0
filterpy==1.4.5
flatbuffers==25.2.10
fonttools==4.59.0
frozenlist==1.7.0
fsspec==2025.7.0
ftfy==6.3.1
future==1.0.0
gfpgan==1.3.8
gguf==0.17.1
gitdb==4.0.12
gitpython==3.1.45
google-ai-generativelanguage==0.6.15
google-api-core==2.25.1
google-api-python-client==2.177.0
google-auth==2.40.3
google-auth-httplib2==0.2.0
google-generativeai==0.8.5
googleapis-common-protos==1.70.0
greenlet==3.2.3
griffe==1.9.0
griptape==1.8.0
griptape-black-forest @ git+https://github.com/griptape-ai/griptape-black-forest.git@f06b2874448f6b610d9eb719f18969f9076de30f
griptape-serper-driver-extension @ git+https://github.com/mertdeveci5/griptape-serper-driver-extension.git@37e9a69148ca8837112cfd8e7578f1bb74970a91
grpcio==1.74.0
grpcio-status==1.71.2
h11==0.16.0
h2==4.2.0
hpack==4.1.0
htmldate==1.9.3
httpcore==1.0.9
httplib2==0.22.0
httpx==0.28.1
httpx-sse==0.4.0
huggingface-hub==0.34.1
humanfriendly==10.0
hydra-core==1.3.2
hyperframe==6.1.0
idna==3.10
imageio==2.37.0
imageio-ffmpeg==0.6.0
img2texture @ git+https://github.com/ltdrdata/img2texture.git@d6159abea44a0b2cf77454d3d46962c8b21eb9d3
importlib-metadata==8.7.0
insightface==0.7.3
installer==0.7.0
iopath==0.1.10
jaraco-classes==3.4.0
jax==0.6.2
jaxlib==0.6.2
jinja2==3.1.6
jiter==0.10.0
jmespath==1.0.1
joblib==1.5.1
jsonpatch==1.33
jsonpointer==3.0.0
jsonschema==4.25.0
jsonschema-specifications==2025.4.1
justext==3.0.2
keyring==24.3.1
kiwisolver==1.4.8
kornia==0.8.1
kornia-rs==0.1.9
langchain-core==0.3.72
langchain-text-splitters==0.3.9
langsmith==0.4.8
lark==1.2.2
lazy-loader==0.4
librosa==0.11.0
llama-cpp-python==0.3.14
llvmlite==0.44.0
lmdb==1.7.3
lxml==5.4.0
lxml-html-clean==0.4.2
mail-parser==4.1.4
mako==1.3.10
markdown==3.8.2
markdown-it-py==3.0.0
markdownify==1.1.0
markupsafe==3.0.2
marqo==3.16.0
marshmallow==3.26.1
marshmallow-enum==1.5.1
matplotlib==3.10.3
matrix-nio==0.25.2
mcp==1.12.2
mdurl==0.1.2
mediapipe==0.10.21
ml-dtypes==0.5.3
more-itertools==10.7.0
moviepy==2.2.1
mpmath==1.3.0
msgpack==1.1.1
mss==10.0.0
multidict==6.6.3
natsort==8.4.0
ndg-httpsclient==0.5.1
networkx==3.4.2
numba==0.61.2
numpy==1.26.4
nvidia-cuda-runtime-cu12==12.9.79
olefile==0.47
ollama==0.5.1
omegaconf==2.3.0
onnx==1.18.0
onnxruntime==1.22.1
open-clip-torch==3.0.0
openai==1.97.1
openai-agents==0.2.4
openapi-pydantic==0.5.1
opencv-contrib-python==4.11.0.86
opencv-python==4.12.0.88
opencv-python-headless==4.12.0.88
opensearch-py==3.0.0
opentelemetry-api==1.35.0
opentelemetry-exporter-otlp-proto-common==1.35.0
opentelemetry-exporter-otlp-proto-http==1.35.0
opentelemetry-instrumentation==0.56b0
opentelemetry-instrumentation-threading==0.56b0
opentelemetry-proto==1.35.0
opentelemetry-sdk==1.35.0
opentelemetry-semantic-conventions==0.56b0
opt-einsum==3.4.0
orjson==3.11.1
packaging==24.2
pandas==2.3.1
peft==0.16.0
pexpect==4.9.0
pgvector==0.4.1
piexif==1.1.3
pilgram==1.2.1
pillow==11.3.0
pinecone==7.3.0
pinecone-plugin-assistant==1.7.0
pinecone-plugin-interface==0.0.7
pip==25.1.1
pkginfo==1.12.1.2
platformdirs==4.3.8
playwright==1.54.0
poetry==1.8.5
poetry-core==1.9.1
poetry-plugin-export==1.8.0
polygraphy==0.49.26
pooch==1.8.2
portalocker==3.2.0
prettytable==3.16.0
primp==0.15.0
proglog==0.1.12
propcache==0.3.2
proto-plus==1.26.1
protobuf==4.25.8
psutil==7.0.0
psycopg2-binary==2.9.10
ptyprocess==0.7.0
pusher==3.3.3
py360convert==1.0.4
pyasn1==0.6.1
pyasn1-modules==0.4.2
pycparser==2.22
pycryptodome==3.23.0
pydantic==2.11.7
pydantic-core==2.33.2
pydantic-settings==2.10.1
pydub==0.25.1
pyee==13.0.0
pygithub==2.6.1
pygments==2.19.2
pyjwt==2.10.1
pymatting==1.1.14
pymongo==4.13.2
pynacl==1.5.0
pyopenssl==25.1.0
pyparsing==3.2.3
pypdf==5.9.0
pyperclip==1.9.0
pyproject-hooks==1.2.0
pyreadline3==3.5.4
python-dateutil==2.9.0.post0
python-dotenv==1.1.1
python-multipart==0.0.20
python-socks==2.7.1
pytz==2025.2
pywin32==311
pywin32-ctypes==0.2.3
pyyaml==6.0.2
qdrant-client==1.15.0
rapidfuzz==3.13.0
realesrgan==0.3.0
redis==6.2.0
referencing==0.36.2
regex==2024.11.6
rembg==2.0.67
requests==2.32.4
requests-toolbelt==1.0.0
rich==14.1.0
rich-rst==1.3.1
rpds-py==0.26.0
rsa==4.9.1
s3transfer==0.13.1
safetensors==0.5.3
sam-2 @ git+https://github.com/facebookresearch/sam2@2b90b9f5ceec907a1c18123530e92e794ad901a4
schema==0.7.7
scikit-image==0.25.2
scikit-learn==1.7.1
scipy==1.15.3
segment-anything==1.0
sentencepiece==0.2.0
setuptools==80.9.0
shellingham==1.5.4
simsimd==6.5.0
six==1.17.0
smmap==5.0.2
sniffio==1.3.1
snowflake==1.6.0
snowflake-connector-python==3.16.0
snowflake-core==1.6.0
snowflake-legacy==1.0.1
snowflake-sqlalchemy==1.7.6
sortedcontainers==2.4.0
sounddevice==0.5.2
soundfile==0.13.1
soupsieve==2.7
soxr==0.5.0.post1
spandrel==0.4.1
sqlalchemy==2.0.42
sse-starlette==3.0.2
starlette==0.47.2
stringzilla==3.12.5
sympy==1.14.0
tavily-python==0.7.10
tb-nightly==2.21.0a20250729
tenacity==9.1.2
tensorboard-data-server==0.7.2
tensorrt==10.13.0.35
tensorrt-cu12==10.13.0.35
tensorrt-cu12-bindings==10.13.0.35
tensorrt-cu12-libs==10.13.0.35
threadpoolctl==3.6.0
tifffile==2025.5.10
tiktoken==0.9.0
timm==1.0.19
tld==0.13.1
tokenizers==0.21.4
toml==0.10.2
tomli==2.2.1
tomlkit==0.13.3
torch==2.7.1+cu128
torchaudio==2.7.1+cu128
torchsde==0.2.6
torchvision==0.22.1+cu128
tqdm==4.67.1
trafilatura==2.0.0
trampoline==0.1.2
transformers==4.54.1
trove-classifiers==2025.5.9.12
typer==0.16.0
types-requests==2.32.4.20250611
typing-extensions==4.14.1
typing-inspection==0.4.1
tzdata==2025.2
tzlocal==5.3.1
unpaddedbase64==2.1.0
uritemplate==4.2.0
urllib3==2.5.0
uuid6==2025.0.1
uv==0.8.3
uvicorn==0.35.0
virtualenv==20.32.0
voyageai==0.3.4
watchdog==6.0.0
wcwidth==0.2.13
webcolors==24.11.1
websockets==15.0.1
werkzeug==3.1.3
wheel==0.45.1
wrapt==1.17.2
xformers==0.0.31.post1
yapf==0.43.0
yarl==1.20.1
zipp==3.23.0
zstandard==0.23.0
```
### Debug Logs
```powershell
Adding extra search path checkpoints D:\hal\stable-diffusion\StabilityMatrix\Models\StableDiffusion
Adding extra search path diffusers D:\hal\stable-diffusion\StabilityMatrix\Models\Diffusers
Adding extra search path loras D:\hal\stable-diffusion\StabilityMatrix\Models\Lora
Adding extra search path loras D:\hal\stable-diffusion\StabilityMatrix\Models\LyCORIS
Adding extra search path clip D:\hal\stable-diffusion\StabilityMatrix\Models\TextEncoders
Adding extra search path clip_vision D:\hal\stable-diffusion\StabilityMatrix\Models\ClipVision
Adding extra search path embeddings D:\hal\stable-diffusion\StabilityMatrix\Models\Embeddings
Adding extra search path vae D:\hal\stable-diffusion\StabilityMatrix\Models\VAE
Adding extra search path vae_approx D:\hal\stable-diffusion\StabilityMatrix\Models\ApproxVAE
Adding extra search path controlnet D:\hal\stable-diffusion\StabilityMatrix\Models\ControlNet
Adding extra search path controlnet D:\hal\stable-diffusion\StabilityMatrix\Models\T2IAdapter
Adding extra search path gligen D:\hal\stable-diffusion\StabilityMatrix\Models\GLIGEN
Adding extra search path upscale_models D:\hal\stable-diffusion\StabilityMatrix\Models\ESRGAN
Adding extra search path upscale_models D:\hal\stable-diffusion\StabilityMatrix\Models\RealESRGAN
Adding extra search path upscale_models D:\hal\stable-diffusion\StabilityMatrix\Models\SwinIR
Adding extra search path hypernetworks D:\hal\stable-diffusion\StabilityMatrix\Models\Hypernetwork
Adding extra search path ipadapter D:\hal\stable-diffusion\StabilityMatrix\Models\IpAdapter
Adding extra search path ipadapter D:\hal\stable-diffusion\StabilityMatrix\Models\IpAdapters15
Adding extra search path ipadapter D:\hal\stable-diffusion\StabilityMatrix\Models\IpAdaptersXl
Adding extra search path prompt_expansion D:\hal\stable-diffusion\StabilityMatrix\Models\PromptExpansion
Adding extra search path ultralytics D:\hal\stable-diffusion\StabilityMatrix\Models\Ultralytics
Adding extra search path ultralytics_bbox D:\hal\stable-diffusion\StabilityMatrix\Models\Ultralytics\bbox
Adding extra search path ultralytics_segm D:\hal\stable-diffusion\StabilityMatrix\Models\Ultralytics\segm
Adding extra search path sams D:\hal\stable-diffusion\StabilityMatrix\Models\Sams
Adding extra search path diffusion_models D:\hal\stable-diffusion\StabilityMatrix\Models\DiffusionModels
Checkpoint files will always be loaded safely.
Total VRAM 12227 MB, total RAM 49069 MB
pytorch version: 2.7.1+cu128
xformers version: 0.0.31.post1
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 5070 : cudaMallocAsync
Using xformers attention
Python version: 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
ComfyUI version: 0.3.47
ComfyUI frontend version: 1.23.4
[Prompt Server] web root: D:\hal\stable-diffusion\StabilityMatrix\Packages\ComfyUI2\venv\lib\site-packages\comfyui_frontend_package\static
Skipping loading of custom nodes
Context impl SQLiteImpl.
Will assume non-transactional DDL.
No target revision found.
Starting server
To see the GUI go to: http://0.0.0.0:8188
got prompt
Using xformers attention in VAE
Using xformers attention in VAE
VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
Using scaled fp8: fp8 matrix mult: False, scale input: False
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load WanTEModel
loaded completely 9511.8 6419.477203369141 True
model weight dtype torch.float16, manual cast: None
model_type FLOW
Requested to load WAN22
loaded partially 8505.368999891358 8505.363647460938 0
CUDA error (C:/a/xformers/xformers/third_party/flash-attention/hopper\flash_fwd_launch_template.h:188): invalid argument
0%| | 0/20 [00:00
Contributor guide
Assessment
This issue has not been assessed yet.