intel / intel/torch-xpu-ops

[upstream_ut] test_torch.py nondeterministic_alert failures: XPU backward kernels raise stale non-deterministic error

Open
#5,304 0 comments 0 reactions 0 assignees View on GitHub
test: ut
Dominant language
Python
Stars
113
Forks
128
Avg merge
5d 9h
Merged PRs (30d)
112

Description

## Description
25 `test_nondeterministic_alert_*` cases in `test/test_torch.py` fail on XPU. The tests do NOT expect a non-determinism error in the default (non-deterministic) mode, but the XPU backward kernels still raise:

RuntimeError: _backward_xpu does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True)'...

producing: "did not expect non-deterministic error message, but got one anyway".

This resembles previously fixed issues (#3117 stale alertNotDeterministic on scatter kernels, #2512/#2516/#2517) and looks like a stale/over-broad `alertNotDeterministic` on additional ops.

## Affected tests
```
test_nondeterministic_alert_AdaptiveAvgPool2d_xpu
test_nondeterministic_alert_AdaptiveAvgPool3d_xpu
test_nondeterministic_alert_AdaptiveMaxPool2d_xpu
test_nondeterministic_alert_AvgPool3d_xpu
test_nondeterministic_alert_MaxPool3d_xpu
test_nondeterministic_alert_FractionalMaxPool2d_xpu
test_nondeterministic_alert_FractionalMaxPool3d_xpu
test_nondeterministic_alert_ReflectionPad1d_xpu
test_nondeterministic_alert_ReflectionPad3d_xpu
test_nondeterministic_alert_ReplicationPad1d_xpu
test_nondeterministic_alert_ReplicationPad2d_xpu
test_nondeterministic_alert_ReplicationPad3d_xpu
test_nondeterministic_alert_interpolate_bilinear_xpu
test_nondeterministic_alert_interpolate_bicubic_xpu
test_nondeterministic_alert_interpolate_linear_xpu
test_nondeterministic_alert_interpolate_trilinear_xpu
test_nondeterministic_alert_grid_sample_2d_xpu
test_nondeterministic_alert_grid_sample_3d_xpu
test_nondeterministic_alert_CTCLoss_xpu
test_nondeterministic_alert_NLLLoss_xpu
test_nondeterministic_alert_EmbeddingBag_max_xpu
test_nondeterministic_alert_bincount_xpu
test_nondeterministic_alert_histc_xpu_float32
test_nondeterministic_alert_median_xpu_float64
test_nondeterministic_alert_put_accumulate_xpu
```

## Error log (example)
```
AssertionError: did not expect non-deterministic error message, but got one anyway: "adaptive_max_pool2d_backward_xpu does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True)'..."
```

## Reproduce
Using pytorch/pytorch#196421 on a PVC machine:
```
python test/test_torch.py TestTorchDeviceTypeXPU.test_nondeterministic_alert_AdaptiveMaxPool2d_xpu
```

## Expected
Match CUDA behavior: these ops should not raise a non-determinism error in the default mode (they should either be deterministic or not flagged when use_deterministic_algorithms is False).

_Filed with AI assistance; reproduced per the details above._

Contributor guide

Open the contributing guide

Research direction

Run the listed TestTorchDeviceTypeXPU case from test/test_torch.py on a PVC machine to confirm the failure. Then trace the affected XPU backward kernels and their alertNotDeterministic usage, comparing the related fixes in issues #3117, #2512, #2516, and #2517. Done means the 25 listed tests match CUDA behavior without unexpected nondeterminism errors in default mode.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.