intel / intel/torch-xpu-ops

[UT][quantization] Deprecated tests not applicable for XPU generalization.

Open
#5,362 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
113
Forks
128
Avg merge
5d 9h
Merged PRs (30d)
112

Description

## Summary

The following upstream `test/quantization/core/*` test files target vendor-specific backends or hardware features that are not available on XPU or are deprecated. They are marked "Not Applicable" for XPU CI and are tracked here so the reason is documented in one place.

## Files

| File | Reason not applicable on XPU |
|------|------------------------------|
| test/quantization/core/test_quantized_tensor.py | Not launched in PyTorch CI & `torch.ao.quantization` deprecated. |
| test/quantization/fx/test_numeric_suite_fx.py | Not launched in PyTorch CI & `torch.ao.quantization` deprecated. |
| test/quantization/core/test_workflow_module.py | Not launched in PyTorch CI & `torch.ao.quantization` deprecated. |
| test/quantization/core/test_workflow_ops.py | Not launched in PyTorch CI & `torch.ao.quantization` deprecated. |

## Notes

- `test_quantized_tensor.py` - this test file theoretically could be generalized and uses accelerator. The problem is that 28 out of 30 device-related tests fail on XPU due to lack of `QuantizedXPU` dispatch key that is deprecated and won't be supported on XPU. Further investigation showed that those `test/quantization/*` tests are not launched anywhere in the PyTorch CI, so they are not tested even on CUDA. That said, it looks like there is no point in generalizing tests that are not even launched on CUDA.

## Why `test/quantization/*` tests are not launched in CI

- First of all the `quantization` pattern is added to the blocklist in the `discover_tests()` function [HERE](https://github.com/pytorch/pytorch/blob/446a293063cfd790bf7adf9b5aecc08597e4c1ae/tools/testing/discover_tests.py#L82), so those tests are not even discovered. The only entry point is through the `test_quantization.py` test file that imports other quantization tests.
- All the `test/run_test.py` calls have the `--exclude-quantization-tests` that excludes quantization tests.
- Another way to launch quantization tests was the `quantization` test config in the workflow `.yml` file ([HERE](https://github.com/pytorch/pytorch/blob/446a293063cfd790bf7adf9b5aecc08597e4c1ae/.ci/pytorch/test.sh#L2464-L2465)), but the last workflow that has `quantization` config was removed several moths ago in https://github.com/pytorch/pytorch/pull/182878

## Quantization tests deprecation

Both `torch.ao.quantized` and creating quantized tensors, along with using `torch.quint8`, `torch.qint8`, and `torch.qint32` dtypes are deprecated. Details in the linked PRs:
- https://github.com/pytorch/pytorch/pull/153892
- https://github.com/pytorch/pytorch/pull/184984

All the listed test files are based on or heavily uses either deprecated `torch.ao.quantized` or creates quantized tensors with deprecated dtypes.

Taking the above into consideration, there is no point in generalizing those quantization test files as they are both not launched in PyTorch CI and use deprecated features and APIs.

Contributor guide

Open the contributing guide

Research direction

Start with the four listed files under test/quantization, then inspect discover_tests() in tools/testing/discover_tests.py, the --exclude-quantization-tests calls in test/run_test.py, and the quantization workflow entry in .ci/pytorch/test.sh. Verify the documented CI and deprecation rationale against these locations; done means the XPU non-applicability record is complete and accurate.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, testing-qa
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.