Comfy-Org / Comfy-Org/ComfyUI

ComfyUI startup crash on ROCm torch 2.5.1+rocm6.2 with comfy_kitchen 0.2.28 (torch custom op schema infer_schema: kernel_size list[int])

Open
#15,441 9 comments 3 reactions 0 assignees View on GitHub
Potential Bug
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 6h
Merged PRs (30d)
155

Description

### Custom Node Testing

- [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Expected Behavior

Environment

GPU: RX 7600
ROCm/hip: 6.2.41133-dd7f95766 (rocm 6.2)
PyTorch: torch==2.5.1+rocm6.2
Python: 3.12
ComfyUI commit: fe4195f7f4275f2626cbafc703acc3ddde1e5490
comfy_kitchen: comfy_kitchen==0.2.28

Crash details
ComfyUI crashes during import/startup when comfy_kitchen registers a PyTorch custom op via `torch.library.custom_op

Top-level error:
ValueError: infer_schema(func): Parameter kernel_size has unsupported type list[int]

Failing function signature:
`(q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, kernel_size: list[int], is_causal: list[bool], scale: float | None) -> torch.Tensor)`

Failing decorator location (from traceback):
/comfy_kitchen/backends/eager/na.py on:
@torch.library.custom_op("comfy_kitchen::na3d", mutates_args=())

Full traceback (as reported)
text

Traceback (most recent call last):
File "/mnt/Docker/ComfyUI/main.py", line 237, in
import comfy.utils
File "/mnt/Docker/ComfyUI/comfy/utils.py", line 25, in
import comfy.memory_management
File "/mnt/Docker/ComfyUI/comfy/memory_management.py", line 8, in
from comfy.quant_ops import QuantizedTensor
File "/mnt/Docker/ComfyUI/comfy/quant_ops.py", line 23, in
import comfy_kitchen as ck
File "/mnt/Docker/comfyenv/lib/python3.12/site-packages/comfy_kitchen/__init__.py", line 3, in
from .backends import cuda as _cuda_backend # noqa: F401
File "/mnt/Docker/comfyenv/lib/python3.12/site-packages/comfy_kitchen/backends/cuda/__init__.py", line 161, in
from comfy_kitchen.backends.eager import rope as _eager_rope # noqa: E402
File "/mnt/Docker/comfyenv/lib/python3.12/site-packages/comfy_kitchen/backends/eager/__init__.py", line 71, in
from .na import na3d
File "/mnt/Docker/comfyenv/lib/python3.12/site-packages/comfy_kitchen/backends/eager/na.py", line 163, in
@torch.library.custom_op("comfy_kitchen::na3d", mutates_args=())
File "/mnt/Docker/comfyenv/lib/python3.12/site-packages/torch/_library/custom_ops.py", line 133, in inner
schema_str = torch.library.infer_schema(fn, mutates_args=mutates_args)
File "/mnt/Docker/comfyenv/lib/python3.12/site-packages/torch/_library/infer_schema.py", line 106, in infer_schema
error_fn(
File "/mnt/Docker/comfyenv/lib/python3.12/site-packages/torch/_library/infer_schema.py", line 58, in error_fn
raise ValueError(
ValueError: infer_schema(func): Parameter kernel_size has unsupported type list[int]. The valid types are: dict_keys([...]). Got func with signature (q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, kernel_size: list[int], is_causal: list[bool], scale: float | None) -> torch.Tensor).

Reproduction steps

Install PyTorch 2.5.1+rocm6.2
Install comfy_kitchen==0.2.28
Start ComfyUI at commit fe4195f7f4275f2626cbafc703acc3ddde1e5490
ComfyUI crashes immediately at import due to torch.library.custom_op schema inference rejection of `kernel_size: list[int]

Expected behavior

ComfyUI should start normally with comfy_kitchen 0.2.28 on ROCm PyTorch 2.5.1.

### Actual Behavior

Actual behavior

ComfyUI fails at startup due to PyTorch custom-op schema inference error for comfy_kitchen::na3d.

### Steps to Reproduce

Update ComfyUI through Comfy Manager

### Debug Logs

```powershell
The trace is posted above. Logs have been overwritten after reinstalling ComfyUI
```

### Other

Had to force 0.2.27 comfy_kitchen to get the app to run.

Contributor guide

Open the contributing guide

Research direction

Start with comfy/quant_ops.py, which imports comfy_kitchen, then inspect the installed comfy_kitchen backends/eager/na.py decorator shown in the traceback. Reproduce the import crash with PyTorch 2.5.1+rocm6.2 and comfy_kitchen 0.2.28, comparing behavior with 0.2.27. Done means ComfyUI starts normally on the reported environment.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.