[XPU] comfy_kitchen FP8 quantized tensors segfault on .to('xpu') during clone()
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
# [XPU] comfy_kitchen FP8 quantized tensors segfault on .to('xpu') during clone()
## Summary
The `comfy_kitchen` FP8 quantized tensor implementation segfaults on Intel XPU when a model containing quantized parameters is moved to the XPU device via `.to('xpu')`. The crash occurs inside `comfy_kitchen.tensor.base.clone()` during the `_quantized_apply` path triggered by `nn.Module._apply()`. This blocks all FP8 quantized text encoders (e.g., Qwen 2.5 VL 7B FP8) from running on Intel GPU.
## Environment
- ComfyUI version: v0.22.0-124-gc87384f2 (commit `c87384f2b8f9e9a7f717e1a269b860698b6634bf`, 2026-06-02)
- comfy-kitchen: 0.2.8
- PyTorch: 2.11.0+xpu
- Python: 3.13.3
- GPU: Intel(R) Graphics [0xe211] (PCI 8086:E211), 160 CUs, 24.4 GB VRAM
- Driver: xe kernel driver, Linux 6.14.0-1008-intel
## Bug Details
### Code path
```
clip.patcher.model.to('xpu')
→ torch.nn.Module._apply() (recursive)
→ comfy.ops._quantized_apply() # ops.py:1315
→ comfy.ops._quantized_apply._quantized_apply_fn() # ops.py:1064
→ torch.nn.Parameter.__new__()
→ comfy_kitchen QTensor.__torch_dispatch__()
→ QTensor._handle_detach() # base.py:396
→ QTensor._copy_with() # base.py:252
→ QTensor.clone() # base.py:73 → SEGFAULT
```
When `nn.Module.to(device)` is called, PyTorch iterates all parameters and calls `_apply()` on each. For quantized parameters managed by `comfy_kitchen`, ComfyUI routes through `_quantized_apply` (`comfy/ops.py:1315`), which calls the quantized tensor's `clone()` to copy data to the new device. The `clone()` implementation in `comfy_kitchen/tensor/base.py:73` crashes on Intel XPU.
### Root cause
The `comfy_kitchen` quantized tensor `clone()` operation attempts to clone FP8 (e4m3fn) quantized data on the XPU device. The underlying operation (likely a raw memory copy or type-specific clone) lacks a working XPU/SYCL kernel and segfaults.
## Reproducer
```python
import comfy.sd
clip = comfy.sd.load_clip(
["qwen_2.5_vl_7b_fp8_scaled.safetensors"],
clip_type=comfy.sd.CLIPType.QWEN_IMAGE,
)
# SEGFAULT here — during .to('xpu') on quantized parameters
clip.patcher.model.to('xpu')
```
Full reproducer: [repro2_clip_quantized_xpu.py](https://github.com/Comfy-Org/ComfyUI/blob/main/xpu-bug-investigation/repro2_clip_quantized_xpu.py)
## Results
**3/3 deterministic segfaults** (exit code 139):
Stack trace from faulthandler:
```
Fatal Python error: Segmentation fault
Current thread (most recent call first):
File "comfy_kitchen/tensor/base.py", line 73 in clone
File "comfy_kitchen/tensor/base.py", line 252 in _copy_with
File "comfy_kitchen/tensor/base.py", line 396 in _handle_detach
File "comfy_kitchen/tensor/base.py", line 340 in __torch_dispatch__
File "torch/nn/parameter.py", line 60 in __new__
File "comfy/ops.py", line 1064 in _quantized_apply
File "comfy/ops.py", line 1315 in _apply
File "torch/nn/modules/module.py", line 934 in _apply
... (recursive _apply calls)
File "torch/nn/modules/module.py", line 1384 in to
File "repro2_clip_quantized_xpu.py", line 69 in main
```
## Impact
- All FP8 quantized text encoders (Qwen 2.5 VL 7B FP8, etc.) must run on CPU on Intel XPU systems
- This blocks end-to-end XPU pipelines — the text encoder becomes a bottleneck
- CLIP encoding on CPU adds ~3-10 seconds depending on prompt length and model size
## Suggested investigation
1. **comfy_kitchen**: Check if the `clone()` method in `tensor/base.py:73` has a device-specific code path. The FP8 quantized tensor likely needs a dequantize→copy→requantize path for cross-device operations, or a raw buffer clone that works on XPU.
2. **PyTorch XPU**: The crash may be in a low-level memory copy kernel for FP8 tensors on the SYCL backend. Check if `torch._C._nn._copy_from` or similar functions support FP8 on XPU.
3. **Workaround**: Dequantize the CLIP model to bf16 before moving to XPU. This increases memory usage but avoids the clone path.
## Related
- ComfyUI's `comfy/ops.py` has a `_quantized_apply` function (line 1064) specifically for handling quantized tensors during `.to()` and `.half()` operations
- The `comfy_kitchen` package (v0.2.8) is ComfyUI's quantization support library
- This issue was discovered during Intel XPU migration of the Qwen-Image-2512 workflow
Contributor guide
Research direction
Start by running xpu-bug-investigation/repro2_clip_quantized_xpu.py with the listed FP8 model and confirm the deterministic crash. Read comfy/ops.py around _quantized_apply and comfy_kitchen/tensor/base.py around clone() and _copy_with(); compare the failing XPU path with supported device behavior. Done means moving the quantized model to XPU completes without a segmentation fault and the reproducer runs successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100