Comfy-Org / Comfy-Org/ComfyUI

UR error during ComfyUI's tutorial with Stable Diffusion model on Ubuntu with Intel GPU TigerLake-H GT1

Open
#6,707 3 comments 0 reactions 0 assignees View on GitHub
User Support
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Your question

Hello,

I get an error, `UR error`, when trying to run [ComfyUI's tutorial](https://docs.comfy.org/get_started/gettingstarted) with the [Stable Diffusion model](https://huggingface.co/Comfy-Org/stable-diffusion-v1-5-archive/resolve/main/v1-5-pruned-emaonly-fp16.safetensors?download=true).

I have checked that Pytorch can use my GPU. E.g., the following Python code works just fine:
```python
import torch
torch.manual_seed(0)
x = torch.rand(47000, 47000, dtype=torch.float32, device='xpu')
```
touch I have to set `UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS=1` to be able to allocate >= 4Gb.

Running `torch.xpu.get_device_capability()` gives
```
{'driver_version': '1.3.29735+27', 'gpu_eu_count': 32, 'gpu_subslice_count': 2, 'has_atomic64': True, 'has_bfloat16_conversions': False, 'has_fp16': True, 'has_fp64': False, 'has_subgroup_2d_block_io': False, 'has_subgroup_matrix_multiply_accumulate': False, 'has_subgroup_matrix_multiply_accumulate_tensor_float32': False, 'max_compute_units': 32, 'max_num_sub_groups': 64, 'max_work_group_size': 512, 'name': 'Intel(R) UHD Graphics', 'platform_name': 'Intel(R) oneAPI Unified Runtime over Level-Zero', 'sub_group_sizes': [8, 16, 32], 'total_memory': 30887309312, 'type': 'gpu', 'vendor': 'Intel(R) Corporation', 'version': '12.0.0'}
```

The error seems to indicate an incompatibility with my GPU, but I am not sure what else I can try to fix it. I have already checked to install the required drivers and dependencies, and have also reported the issue at https://github.com/intel/intel-extension-for-pytorch/issues/781.

Any tips would be very much appreciated.

### Logs

```powershell
Checkpoint files will always be loaded safely.
[W204 15:12:02.899304886 OperatorEntry.cpp:155] Warning: Warning only once for all operators, other operators may also be overridden.
Overriding a previously registered kernel for the same operator and the same dispatch key
operator: aten::_cummax_helper(Tensor self, Tensor(a!) values, Tensor(b!) indices, int dim) -> ()
registered at /build/pytorch/build/aten/src/ATen/RegisterSchema.cpp:6
dispatch key: XPU
previous kernel: registered at /build/pytorch/build/aten/src/ATen/RegisterCPU.cpp:30476
new kernel: registered at /build/intel-pytorch-extension/build/Release/csrc/gpu/csrc/aten/generated/ATen/RegisterXPU.cpp:2971 (function operator())
Total VRAM 29456 MB, total RAM 31815 MB
pytorch version: 2.5.1+cxx11.abi
Set vram state to: NORMAL_VRAM
Device: xpu
Using pytorch attention
ComfyUI version: 0.3.13
[Prompt Server] web root: ComfyUI/web

Import times for custom nodes:
0.0 seconds: ComfyUI/custom_nodes/websocket_image_save.py

Starting server

To see the GUI go to: http://127.0.0.1:8188
got prompt
model weight dtype torch.float16, manual cast: None
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE
VAE load device: xpu:0, offload device: cpu, dtype: torch.bfloat16
CLIP/text encoder model load device: xpu:0, offload device: cpu, current: cpu, dtype: torch.float16
loaded diffusion model directly to GPU
Requested to load BaseModel
loaded completely 9.5367431640625e+25 1639.406135559082 True
2025-02-04 15:12:09,982 - _logger.py - IPEX - INFO - Currently split master weight for xpu only support sgd
2025-02-04 15:12:10,002 - _logger.py - IPEX - INFO - Conv BatchNorm folding failed during the optimize process.
2025-02-04 15:12:10,021 - _logger.py - IPEX - INFO - Linear BatchNorm folding failed during the optimize process.
Requested to load SD1ClipModel
loaded completely 26585.26240234375 235.84423828125 True
2025-02-04 15:12:10,206 - _logger.py - IPEX - INFO - Currently split master weight for xpu only support sgd
2025-02-04 15:12:10,211 - _logger.py - IPEX - INFO - Conv BatchNorm folding failed during the optimize process.
2025-02-04 15:12:10,214 - _logger.py - IPEX - INFO - Linear BatchNorm folding failed during the optimize process.
!!! Exception during processing !!! UR error
Traceback (most recent call last):
File "ComfyUI/execution.py", line 327, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/execution.py", line 202, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/execution.py", line 174, in _map_node_over_list
process_inputs(input_dict, i)
File "ComfyUI/execution.py", line 163, in process_inputs
results.append(getattr(obj, func)(**inputs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/nodes.py", line 69, in encode
return (clip.encode_from_tokens_scheduled(tokens), )
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfy/sd.py", line 148, in encode_from_tokens_scheduled
pooled_dict = self.encode_from_tokens(tokens, return_pooled=return_pooled, return_dict=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfy/sd.py", line 210, in encode_from_tokens
o = self.cond_stage_model.encode_token_weights(tokens)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfy/sd1_clip.py", line 635, in encode_token_weights
out = getattr(self, self.clip).encode_token_weights(token_weight_pairs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfy/sd1_clip.py", line 45, in encode_token_weights
o = self.encode(to_encode)
^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfy/sd1_clip.py", line 252, in encode
return self(tokens)
^^^^^^^^^^^^
File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfy/sd1_clip.py", line 224, in forward
outputs = self.transformer(tokens, attention_mask_model, intermediate_output=self.layer_idx, final_layer_norm_intermediate=self.layer_norm_hidden_state, dtype=torch.float32)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfy/clip_model.py", line 137, in forward
x = self.text_model(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfy/clip_model.py", line 101, in forward
x = self.embeddings(input_tokens, dtype=dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfy/clip_model.py", line 82, in forward
return self.token_embedding(input_tokens, out_dtype=dtype) + comfy.ops.cast_to(self.position_embedding.weight, dtype=dtype, device=input_tokens.device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfy/ops.py", line 209, in forward
return self.forward_comfy_cast_weights(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "ComfyUI/comfy/ops.py", line 205, in forward_comfy_cast_weights
raise error
File "ComfyUI/comfy/ops.py", line 201, in forward_comfy_cast_weights
return torch.nn.functional.embedding(input, weight, self.padding_idx, self.max_norm, self.norm_type, self.scale_grad_by_freq, self.sparse).to(dtype=output_dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: UR error
```

### Other

### Setup

* Ubuntu 24.04.1 LTS
* Python `3.12`
* ComfyUI `v0.0.2-736-g8d88bfaf`
* GPU: Intel(R) UHD Graphics, TigerLake-H GT1

```bash
# based on https://dgpu-docs.intel.com/driver/client/overview.html#installing-client-gpus-on-ubuntu-desktop-24-04-lts

wget -qO - https://repositories.intel.com/gpu/intel-graphics.key | \
sudo gpg --yes --dearmor --output /usr/share/keyrings/intel-graphics.gpg

echo "deb [arch=amd64,i386 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu noble client" | \
sudo tee /etc/apt/sources.list.d/intel-gpu-noble.list

sudo apt update

apt-get install -y libze-intel-gpu1 libze1 intel-opencl-icd clinfo intel-gsc
apt-get install -y libze-dev intel-ocloc
# missing in the instructions, unsure if needed
apt-get install -y intel-level-zero-gpu
```

After cloning the ComfyUI repository:
```bash
# python
pyenv install 3.12
pyenv local 3.12

# create virtual env
python -m venv comfyui-venv
source comfyui-venv/bin/activate
pip install --upgrade pip

# https://pytorch-extension.intel.com/installation?platform=gpu&version=v2.5.10%20xpu&os=linux/wsl2&package=pip
python -m pip install torch==2.5.1+cxx11.abi torchvision==0.20.1+cxx11.abi torchaudio==2.5.1+cxx11.abi intel-extension-for-pytorch==2.5.10+xpu oneccl_bind_pt==2.5.0+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/

# install other requirements
pip install -r requirements.txt
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.