intel / intel/torch-xpu-ops

[B60][torch-xpu-ops][XPU] STFT complex128 gradient tests hit UR_RESULT_ERROR_DEVICE_LOST

Open
#4,796 3 comments 0 reactions 0 assignees View on GitHub
agent:needs-human agent:skipped dependency component: driver hw: BMG test: ut
Dominant language
Python
Stars
113
Forks
128
Avg merge
5d 9h
Merged PRs (30d)
112

Description

## Summary

B60 shows `UR_RESULT_ERROR_DEVICE_LOST` in torch-xpu-ops STFT complex128 gradient tests. The same tests passed on PVC.

## Failure Category

Device lost during STFT complex128 gradient/autograd execution.

Representative message:

```text
UR_RESULT_ERROR_DEVICE_LOST
```

## Scope

- Test suite: torch-xpu-ops
- Platform comparison: B60 error/failed; PVC passed
- Affected rows observed: 3

## Failures

Direct torch-xpu-ops XPU test files:

```text
test/xpu/test_ops_gradients_xpu.py::TestBwdGradientsXPU::test_fn_grad_stft_xpu_complex128
test/xpu/test_ops_fwd_gradients_xpu.py::TestFwdGradientsXPU::test_fn_fwgrad_bwgrad_stft_xpu_complex128
test/xpu/test_ops_gradients_xpu.py::TestBwdGradientsXPU::test_fn_gradgrad_stft_xpu_complex128
```

All fail with:

```text
UR_RESULT_ERROR_DEVICE_LOST
```

## Expected Behavior

The STFT complex128 gradient paths should complete without device loss, consistent with PVC behavior.

## Notes

All visible rows are STFT + complex128 gradient variants, so this appears to be a focused operator/autograd failure family.

## Full Case List

Total cases: 3

1. `torch-xpu-ops` | `test/xpu/test_ops_gradients_xpu.py::TestBwdGradientsXPU::test_fn_grad_stft_xpu_complex128` | `error`
- `UR_RESULT_ERROR_DEVICE_LOST`
2. `torch-xpu-ops` | `test/xpu/test_ops_fwd_gradients_xpu.py::TestFwdGradientsXPU::test_fn_fwgrad_bwgrad_stft_xpu_complex128` | `failed`
- `UR_RESULT_ERROR_DEVICE_LOST`
3. `torch-xpu-ops` | `test/xpu/test_ops_gradients_xpu.py::TestBwdGradientsXPU::test_fn_gradgrad_stft_xpu_complex128` | `failed`
- `UR_RESULT_ERROR_DEVICE_LOST`

## Reproducer / Environment Setup

```bash
docker run -it -e TZ=Asia/Shanghai --device=/dev/mem --device=/dev/dri --group-add video --privileged -v $(realpath ${HOME}):/home/jenkins --shm-size=8g intelgpu/ubuntu-26.04-rolling:26.18 bash

apt update
apt upgrade -y

curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env
uv venv myvenv --python 3.10 --clear
source myvenv/bin/activate
uv pip install pip wheel "setuptools<81"
uv pip install torch==2.13.0 torchaudio torchvision --index-url https://download.pytorch.org/whl/xpu

pytorch_commit="$(python -c 'import torch; print(torch.version.git_version)')"
git clone https://github.com/pytorch/pytorch
cd pytorch
git checkout ${pytorch_commit}

uv pip install -r .ci/docker/requirements-ci.txt
uv pip install -U typing_extensions
uv pip install pytest pytest-timeout pytest-xdist pytest-rerunfailures

cd third_party/
git clone https://github.com/intel/torch-xpu-ops
xpu_ops_commit="$(cat xpu.txt)"
cd torch-xpu-ops
git checkout $xpu_ops_commit
cd test/xpu

python -m pytest test_ops_gradients_xpu.py -k 'test_fn_grad_stft_xpu_complex128 or test_fn_gradgrad_stft_xpu_complex128'
python -m pytest test_ops_fwd_gradients_xpu.py -k 'test_fn_fwgrad_bwgrad_stft_xpu_complex128'
```

Contributor guide

Open the contributing guide

Research direction

Start with test/xpu/test_ops_gradients_xpu.py and test/xpu/test_ops_fwd_gradients_xpu.py, then run the three pytest commands from the reproducer in the provided XPU environment. Compare the STFT complex128 gradient and autograd failures on B60 with the passing PVC behavior. Done means all three tests complete without UR_RESULT_ERROR_DEVICE_LOST on B60.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.