[OneDNN3.13][BMG] 3 confirmed performance drops
- Dominant language
- Python
- Stars
- 113
- Forks
- 128
- Avg merge
- 5d 9h
- Merged PRs (30d)
- 112
Description
### 🐛 Describe the bug
## Summary
OneDNN3.13 validation on BMG shows 3 confirmed drops compared PT2.14 release test after applying rerun verification.
## Affected cases
Suite | Dtype | Mode | Model | Ind Ratio | Eager Ratio
-- | -- | -- | -- | -- | --
timm_models | float32 | inference | tnt_s_patch16_224 | 0.8586 | 0.9996
## Reproducer
```bash
##download pytorch
gh --repo pytorch/pytorch run download 33480437862 -p "manywheel-py3_10-xpu"
pip install torch==2.14.0 torchvision torchaudio triton-xpu --index-url https://download.pytorch.org/whl/test/xpu
TORCH_COMMIT_ID=$(python -c 'import torch; print(torch.version.git_version)')
git clone https://github.com/pytorch/pytorch
cd pytorch
git checkout ${TORCH_COMMIT_ID}
pip install requests
cd ../
pip install pandas psutil scipy pyyaml
## tim_models
cd pytorch
TIMM_COMMIT_ID="$(cat .ci/docker/ci_commit_pins/timm.txt 2> /dev/null || echo 'v1.0.14')"
pip install git+https://github.com/huggingface/pytorch-image-models@${TIMM_COMMIT_ID}
## torchbench
cd pytorch
TORCHBENCH_COMMIT_ID="$(cat .ci/docker/ci_commit_pins/torchbench.txt 2> /dev/null || echo 'e03a63be')"
cd ../
rm -rf ./benchmark
git clone https://github.com/pytorch/benchmark benchmark
cd benchmark
git checkout ${TORCHBENCH_COMMIT_ID}
pip install -r requirements.txt
python install.py --continue_on_fail phlippe_resnet functorch_dp_cifar10
pip install pyre-extensions
curl -fsSL https://raw.githubusercontent.com/facebookresearch/dlrm/refs/heads/torchrec-dlrm/requirements.txt |xargs pip install
pip uninstall -y fbgemm_gpu_nightly-cpu
pip install git+https://github.com/nocoding03/gym@fix-np
pip install torchao --index-url https://download.pytorch.org/whl/test/xpu
pip install -U numpy==1.26.4
## install target install
pip uninstall torch -y
pip install --no-deps /download/path/torch-*.whl
##run command line
python benchmarks/dynamo/timm_models.py --performance -d xpu -n10 --backend=inductor --cold-start-latency --inference --float32 --only tnt_s_patch16_224
python benchmarks/dynamo/torchbench.py --performance -d xpu -n10 --backend=inductor --cold-start-latency --training --amp --amp-dtype bfloat16 --only phlippe_resnet
python benchmarks/dynamo/torchbench.py --performance -d xpu -n10 --backend=inductor --cold-start-latency --training --float16 --only functorch_dp_cifar10
```
### Versions
torch 2.14.0.dev20260901+xpu
torchao 0.18.0+xpu
torchaudio 2.11.0+xpu
torchvision 0.29.0+xpu
Contributor guide
Assessment
This issue has not been assessed yet.