[Bug]: ARK SageV1 BF16 D128 causes DEVICE_LOST on Windows Arc B580
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 175
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 99
Description
### Problem Description
On Windows with an Intel Arc B580, the smallest BF16 SageV1 call I tested causes the Level Zero device to be lost at the first synchronization. The equivalent PyTorch SDPA call with the same Q/K/V tensors succeeds and returns finite values.
This does not appear to be an OOM: the reproduction uses only [1, 8, 128, 128] BF16 tensors, and a fresh Python process can run a BF16 matrix multiplication successfully immediately afterward.
I reproduced the same failure with the default Level Zero command-list behavior and with SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=0.
### Reproduction Steps
Build ARK for XPU/SYCL-TLA and run:
~~~python
import math
import torch
import torch.nn.functional as F
import auto_round_kernel as ark
torch.manual_seed(20260819)
q = torch.randn((1, 8, 128, 128), device="xpu", dtype=torch.bfloat16)
k = torch.randn_like(q)
v = torch.randn_like(q)
torch.xpu.synchronize()
# Control passes and is finite.
ref = F.scaled_dot_product_attention(q, k, v, dropout_p=0.0, is_causal=False)
torch.xpu.synchronize()
assert torch.isfinite(ref).all()
# The first synchronization after this call loses the device.
out = ark.sagev1(
q, k, v,
dropout_p=0.0,
is_causal=False,
scale=1.0 / math.sqrt(128),
quant_block_size=64,
tensor_layout="HND",
smooth_k=True,
)
torch.xpu.synchronize()
~~~
### Environment Information
- OS: Windows x64, build 26100
- GPU: Intel Arc B580 Graphics, 12 GiB
- GPU driver: 32.0.101.8974 (2026-08-11)
- Python: 3.13.12
- PyTorch: 2.13.0+xpu
- Torch XPU runtime: 20260000
- oneAPI DPC++ compiler: 2026.1.0, build 20260617
- Tested ARK minimal binary source commit: 674377982561707cdc51db99cd8ff634369e20b7
- SYCL-TLA: fedbba404c6fb634554b7e5aec9a463710b6f952 (260630)
- Current AutoRound main checked: b9f3d0079d014c73a1ff009800c597b9bc3f2a36
Current main still pins SYCL-TLA 260630. I also reviewed the newer ARK compiler-memory refactor ff2fd25c644d4c4ea385f2eaeb691fa6eb6edc29; it splits normal SDPA template instantiations but does not change the SageV1 D128 execution path used by this reproduction.
### Error Logs
```shell
RuntimeError: level_zero backend failed with error: 20 (UR_RESULT_ERROR_DEVICE_LOST)
```
### Additional Context
- ARK import succeeds and exposes sagev1, sage, and sage_dynamic_quant.
- PyTorch SDPA with the exact same input shape and dtype succeeds.
- The failure also occurs when immediate command lists are disabled.
- A fresh process after the failure successfully completes a BF16 64x64 XPU matmul, so the GPU/driver recovers and there is no persistent device damage.
- The minimal build isolates BF16 SageV1, head dimension 128, and excludes unrelated kernels to fit compilation within 32 GiB system RAM. The executed SageV1/SYCL-TLA path is the same one still present in current main.
Is Windows + Battlemage G21 + BF16 SageV1 D128 expected to be supported with this version combination? If so, could you advise the supported driver/Torch/oneAPI matrix or point to a kernel fix/guard for this device-lost path?
Contributor guide
Assessment
This issue has not been assessed yet.