intel / intel/torch-xpu-ops

[release/2.14] [Bug Skip]: UT failures 2026-7-19

Open
#4,388 4 comments 0 reactions 1 assignee View on GitHub

@zuzanna1x is already working on this.

Since Jul 20, 2026.

os: Linux os: Windows regression skipped
Dominant language
Python
Stars
113
Forks
129
Avg merge
5d 9h
Merged PRs (30d)
112

Description

🐛 Describe the bug with skip template

Cases:
op_ut,third_party.torch-xpu-ops.test.xpu.test_scaled_matmul_cuda_xpu.TestFP8MatmulXPU,test_blockwise_mxfp8_compile_xpu
op_ut,third_party.torch-xpu-ops.test.xpu.test_scaled_matmul_cuda_xpu.TestFP8MatmulXPU,test_blockwise_nvfp4_compile_xpu
op_ut,third_party.torch-xpu-ops.test.xpu.test_scaled_matmul_cuda_xpu.TestFP8MatmulXPU,test_passed_swizzle_arrays_xpu
op_ut,third_party.torch-xpu-ops.test.xpu.test_custom_ops_xpu.TestCustomOp,test_backward_output_differentiability_non_tensor
op_ut,third_party.torch-xpu-ops.test.xpu.test_torch_xpu.TestTorchDeviceTypeXPU,test_cpp_warnings_have_python_context_xpu

ErrorLog

RuntimeError: Invalid scaling configuration.

Affected Tests:

~~op_ut,third_party.torch-xpu-ops.test.xpu.test_scaled_matmul_cuda_xpu.TestFP8MatmulXPU,test_blockwise_mxfp8_compile_xpu~~
~~op_ut,third_party.torch-xpu-ops.test.xpu.test_scaled_matmul_cuda_xpu.TestFP8MatmulXPU,test_blockwise_nvfp4_compile_xpu~~

______________ TestFP8MatmulXPU.test_blockwise_mxfp8_compile_xpu _______________
[gw0] linux -- Python 3.10.20 /tmp/xpu-tool/Python/3.10.20/x64/bin/python
Traceback (most recent call last):
  File "/__w/torch-xpu-ops/torch-xpu-ops/pytorch/third_party/torch-xpu-ops/test/xpu/../../../../test/test_scaled_matmul_cuda.py", line 2654, in test_blockwise_mxfp8_compile
    C = compiled_scaled_mm(
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1188, in compile_wrapper
    result = fn(*args, **kwargs)
  File "/__w/torch-xpu-ops/torch-xpu-ops/pytorch/third_party/torch-xpu-ops/test/xpu/../../../../test/test_scaled_matmul_cuda.py", line 237, in scaled_mm_wrap
    scale_recipe_a, swizzle_a = infer_scale_swizzle(a, scale_a)
  File "/__w/torch-xpu-ops/torch-xpu-ops/pytorch/third_party/torch-xpu-ops/test/xpu/../../../../test/test_scaled_matmul_cuda.py", line 239, in torch_dynamo_resume_in_scaled_mm_wrap_at_237
    scale_recipe_b, swizzle_b = infer_scale_swizzle(b, scale_b)
  File "/__w/torch-xpu-ops/torch-xpu-ops/pytorch/third_party/torch-xpu-ops/test/xpu/../../../../test/test_scaled_matmul_cuda.py", line 239, in torch_dynamo_resume_in_scaled_mm_wrap_at_239
    scale_recipe_b, swizzle_b = infer_scale_swizzle(b, scale_b)
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1468, in _fn
    return fn(*args, **kwargs)
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1285, in forward
    return compiled_fn(full_args)
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/runtime_wrappers.py", line 1188, in runtime_wrapper
    result = _codegen_runtime_wrapper(
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/codegen.py:codegen(runtime_wrapper_orchestration)", line 8, in _runtime_wrapper
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/runtime_wrappers.py", line 2593, in __call__
    return self.compiled_fn(*args, **kwargs)
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/site-packages/torch/_inductor/output_code.py", line 828, in __call__
    return self.current_callable(inputs)
  File "/tmp/torchinductor_root/m2/cm2ge5kl6bbz5cmfzu3ch4cms42fxmm42fs5ujsep2fm7pqd2ybw.py", line 60, in call
    extern_kernels._scaled_mm(arg2_1, arg3_1, arg0_1, arg1_1, out_dtype=torch.bfloat16, use_fast_accum=False, out=buf0)
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/site-packages/torch/_compile.py", line 54, in inner
    return disable_fn(*args, **kwargs)
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1468, in _fn
    return fn(*args, **kwargs)
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/runtime_wrappers.py", line 477, in __torch_dispatch__
    res = func(*args, **kwargs)
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/site-packages/torch/_ops.py", line 916, in __call__
    return self._op(*args, **kwargs)
RuntimeError: Invalid scaling configuration.
- For TensorWise scaling, a and b should be float8, scales should be float and singletons.
- For RowWise scaling, a and b should be float8, scales should be float, scale_a should be (128, 1) and scale_b should be (1, 128), and both should be contiguous.
- For BlockWise 1x128 scaling, a and b should be float8, scales should be float, scale_a should be (128, 1) and scale_b should be (1, 128).
- For BlockWise 128x128 scaling, a and b should be float8, scales should be float, scale_a should be (1, 1) and scale_b should be (1, 1).
Got a.dtype()=Float8_e4m3fn, scale_a.dtype()=Float8_e8m0fnu, scale_a.size()=[128, 4], scale_a.stride()=[4, 1], b.dtype()=Float8_e4m3fn, scale_b.dtype()=Float8_e8m0fnu, scale_b.size()=[128, 4] and scale_b.stride()=[4, 1]
AssertionError: ValueError not raised

Affected Tests:

op_ut,third_party.torch-xpu-ops.test.xpu.test_scaled_matmul_cuda_xpu.TestFP8MatmulXPU,test_passed_swizzle_arrays_xpu

_______________ TestFP8MatmulXPU.test_passed_swizzle_arrays_xpu ________________
[gw6] linux -- Python 3.10.20 /tmp/xpu-tool/Python/3.10.20/x64/bin/python
Traceback (most recent call last):
  File "/__w/torch-xpu-ops/torch-xpu-ops/pytorch/third_party/torch-xpu-ops/test/xpu/../../../../test/test_scaled_matmul_cuda.py", line 2274, in test_passed_swizzle_arrays
    with self.assertRaisesRegex(
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/unittest/case.py", line 226, in __exit__
    self._raiseFailure("{} not raised".format(exc_name))
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/unittest/case.py", line 163, in _raiseFailure
    raise self.test_case.failureException(msg)
AssertionError: ValueError not raised
AssertionError: RuntimeError not raised

Affected Tests:

________ TestCustomOp.test_backward_output_differentiability_non_tensor ________
[gw7] linux -- Python 3.10.20 /tmp/xpu-tool/Python/3.10.20/x64/bin/python
Traceback (most recent call last):
  File "/__w/torch-xpu-ops/torch-xpu-ops/pytorch/third_party/torch-xpu-ops/test/xpu/test_custom_ops_xpu.py", line 1611, in test_backward_output_differentiability_non_tensor
    with self.assertRaisesRegex(RuntimeError, "is not a Tensor"):
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/unittest/case.py", line 226, in __exit__
    self._raiseFailure("{} not raised".format(exc_name))
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/unittest/case.py", line 163, in _raiseFailure
    raise self.test_case.failureException(msg)
AssertionError: RuntimeError not raised
AssertionError: False is not true

Affected Tests:

op_ut,third_party.torch-xpu-ops.test.xpu.test_torch_xpu.TestTorchDeviceTypeXPU,test_cpp_warnings_have_python_context_xpu

_______ TestTorchDeviceTypeXPU.test_cpp_warnings_have_python_context_xpu _______
[gw3] linux -- Python 3.10.20 /tmp/xpu-tool/Python/3.10.20/x64/bin/python
Traceback (most recent call last):
  File "/__w/torch-xpu-ops/torch-xpu-ops/pytorch/third_party/torch-xpu-ops/test/xpu/test_torch_xpu.py", line 1206, in test_cpp_warnings_have_python_context
    self.assertTrue(
  File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: False is not true

Pytorch Version

latest good : 75432f27c8b35979927020a551d040b187474d7f
current : 09ef78c325438bc1c19a18250c17d8a2ee9fa89f

Versions
Detail Collecting environment information... PyTorch version: 2.14.0a0+git09ef78c Is debug build: False CUDA used to build PyTorch: None ROCM used to build PyTorch: N/A

OS: Ubuntu 24.04.4 LTS (x86_64)
GCC version: (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0
Clang version: Could not collect
CMake version: version 3.31.6
Libc version: glibc-2.39

Python version: 3.10.20 (main, Jun 16 2026, 21:01:32) [GCC 13.3.0] (64-bit runtime)
Python platform: Linux-5.15.0-179-generic-x86_64-with-glibc2.39
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
Is XPU available: True
XPU used to build PyTorch: 20260000
Intel GPU driver version:

  • libze1: 1.24.0.0-1146~24.04
  • intel-opencl-icd: 25.18.33578.51-1146~24.04
    Intel GPU models onboard:
  • Intel(R) Data Center GPU Max 1100
  • Intel(R) Data Center GPU Max 1100
  • Intel(R) Data Center GPU Max 1100
  • Intel(R) Data Center GPU Max 1100
  • Intel(R) Data Center GPU Max 1100
  • Intel(R) Data Center GPU Max 1100
  • Intel(R) Data Center GPU Max 1100
  • Intel(R) Data Center GPU Max 1100
    Intel GPU models detected:
    Vulnerability Retbleed: Not affected
    Vulnerability Spec rstack overflow: Not affected
    Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
    Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
    Vulnerability Spectre v2: Mitigation; Enhanced / Automatic IBRS; IBPB conditional; PBRSB-eIBRS SW sequence; BHI BHI_DIS_S
    Vulnerability Srbds: Not affected
    Vulnerability Tsa: Not affected
    Vulnerability Tsx async abort: Not affected
    Vulnerability Vmscape: Mitigation; IBPB before exit to userspace

Versions of relevant libraries:
[pip3] dpcpp-cpp-rt==2026.0.0
[pip3] impi-rt==2021.18.0
[pip3] intel-cmplr-lib-rt==2026.0.0
[pip3] intel-cmplr-lib-ur==2026.0.0
[pip3] intel-cmplr-lic-rt==2026.0.0
[pip3] intel-opencl-rt==2026.0.0
[pip3] intel-openmp==2026.0.0
[pip3] intel-pti==0.17.0
[pip3] intel-sycl-rt==2026.0.0
[pip3] mkl==2026.0.0
[pip3] mypy==1.16.0
[pip3] mypy_extensions==1.1.0
[pip3] numpy==1.24.4
[pip3] nvidia-cuda-cupti==13.3.75
[pip3] oneccl==2022.0.0
[pip3] oneccl-devel==2022.0.0
[pip3] onemkl-license==2026.0.0
[pip3] onemkl-sycl-blas==2026.0.0
[pip3] onemkl-sycl-dft==2026.0.0
[pip3] onemkl-sycl-lapack==2026.0.0
[pip3] onemkl-sycl-rng==2026.0.0
[pip3] onemkl-sycl-sparse==2026.0.0
[pip3] onnx==1.21.0
[pip3] onnx-ir==0.1.16
[pip3] onnxscript==0.6.2
[pip3] optree==0.13.0
[pip3] tbb==2023.0.0
[pip3] tcmlib==1.5.0
[pip3] torch==2.14.0a0+git09ef78c
[pip3] torchao==0.18.0.dev20260717+xpu
[pip3] torchaudio==2.11.0a0+c0cbdb9
[pip3] torchvision==0.29.0a0+864d526
[pip3] triton-xpu==3.7.2+git5fcc14d9
[pip3] umf==1.1.0
[conda] No relevant packages

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.