intel / intel/llm-scaler

[Windows][BMG] Sol-Attn forward_cute causes DEVICE_LOST on Arc B580 while prepare succeeds

Open
#653 1 comment 0 reactions 1 assignee Claimed by @xiangyuT View on GitHub
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

## Summary

I tested the new BMG Sol-Attn path from llm-scaler commit `3ac20587ed77b3e8adb100ca56fa0fbabb062db5` on Windows 11 with an Intel Arc B580.

The Windows sidecar can be compiled and loaded after two host-portability adjustments, and `omni_xpu_sol_attn.prepare` runs successfully. However, the first `omni_xpu_sol_attn.forward_cute` execution causes a Level Zero device loss, even with the smallest upstream-style BF16 D128 input.

I understand that `setup.py` currently filters CUTE/Sol-Attn out on Windows. This report is intended as a minimal reproduction and a request for guidance on the missing Windows BMG runtime support.

## Environment

- GPU: Intel Arc B580, PCI Device ID `E20B`
- Driver: `32.0.101.8974`
- OS: Windows 11, build `26100`
- Python: `3.13.12`
- PyTorch: `2.13.0+xpu`
- oneAPI DPC++: `2026.1.0 Build 20260617`
- Visual Studio Build Tools: `17.14.37`
- llm-scaler: `3ac20587ed77b3e8adb100ca56fa0fbabb062db5`
- ComfyUI-SolAttn_xpu: `5f1c4aac3ca32a00b0b4c15ddbb7cb53fa43344d`
- SYCL-TLA: `2fc09973bfdf15755090fcb0e3b6ad236408a992`

## Isolated Windows build

I compiled these three sources into a Torch-loadable `.pyd`:

- `cute_fmha_torch.cpp`
- `sol_attn_prepare.cpp`
- `sol_attn_torch.cpp`

The build used the same BMG AOT and SPIR-V extension intent as the Linux CUTE branch:

- `-fsycl-targets=spir64_gen`
- backend target `-device bmg`
- `SPV_INTEL_split_barrier`
- `SPV_INTEL_2d_block_io`
- `SPV_INTEL_subgroup_matrix_multiply_accumulate`

Two build-only portability adjustments were needed:

1. Explicitly initialize the installed VS 2022 Build Tools before oneAPI.
2. In a private include overlay only, replace the host type alias `ushort` with equivalent standard C++ `unsigned short` in `mma_xe_legacy_spirv.hpp`.

The upstream checkouts were not modified. The resulting sidecar loads through `torch.ops.load_library()` and registers both `prepare` and `forward_cute`.

Built sidecar:

- Size: `1,588,736` bytes
- SHA-256: `4BE1F0F50B84CB1AFE8D6DAAF1D9B15051E2E1D04F9A75054C84E8323B363BA3`

## Minimal reproduction

`prepare` completes and synchronizes successfully. The failure occurs only after `forward_cute`.

```python
import torch

torch.ops.load_library(r"")
ops = torch.ops.omni_xpu_sol_attn

# Sanity check passes.
basic = torch.ones((64,), device="xpu", dtype=torch.float32)
torch.xpu.synchronize()
print(basic.sum().item()) # 64.0

shape = (1, 31, 1, 128)
q = torch.zeros(shape, device="xpu", dtype=torch.bfloat16)
k = torch.zeros_like(q)
v = torch.zeros_like(q)
scale = 128 ** -0.5

prepared = ops.prepare(q, k, v, float(scale), 1.0, 0, 0, 0, 0)
torch.xpu.synchronize() # succeeds

print([tuple(t.shape) for t in prepared])
# [(1,1,1,128), (1,1,1,128), (1,1,1,128), (1,1,1), (1,1,1)]

output = ops.forward_cute(q, k, v, *prepared, float(scale))
torch.xpu.synchronize() # DEVICE_LOST
```

Actual error:

```text
RuntimeError: level_zero backend failed with error: 20
(UR_RESULT_ERROR_DEVICE_LOST)
```

An H3-style strided QKV view with shape `[1, 129, 2, 128]` fails in the same way.

After the test process exits, a new Python process can run ordinary XPU tensors normally, so the device recovers and the failure is isolated to the CUTE/DPAS forward launch.

## Expected behavior

The minimal all-zero BF16 D128 input should complete and return a finite all-zero output, as in the upstream Sol-Attn correctness tests.

## Questions

1. Is Windows BMG CUTE/Sol-Attn runtime support currently expected to work, or is Linux the only supported path at this commit?
2. Are different Windows compiler/linker/AOT flags required for the CUTE sidecar?
3. Could this be related to the 256-GRF launch property, Block2D/SPIR-V extensions, or the Windows Level Zero launch path?
4. If useful, I can provide the exact compile command and the isolated source overlay diff.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.