intel / intel/torch-xpu-ops

[OneDNN3.13][BMG] 3 confirmed performance drops

Open
#5,223 2 comments 0 reactions 1 assignee Claimed by @LuFinch View on GitHub
dependency component: oneDNN hw: BMG performance
Dominant language
Python
Stars
113
Forks
128
Avg merge
5d 9h
Merged PRs (30d)
112

Description

### 🐛 Describe the bug

## Summary
OneDNN3.13 validation on BMG shows 3 confirmed drops compared PT2.14 release test after applying rerun verification.

## Affected cases

Suite | Dtype | Mode | Model | Ind Ratio | Eager Ratio
-- | -- | -- | -- | -- | --
timm_models | float32 | inference | tnt_s_patch16_224 | 0.8586 | 0.9996

## Reproducer
```bash
##download pytorch
gh --repo pytorch/pytorch run download 33480437862 -p "manywheel-py3_10-xpu"
pip install torch==2.14.0 torchvision torchaudio triton-xpu --index-url https://download.pytorch.org/whl/test/xpu
TORCH_COMMIT_ID=$(python -c 'import torch; print(torch.version.git_version)')
git clone https://github.com/pytorch/pytorch
cd pytorch
git checkout ${TORCH_COMMIT_ID}
pip install requests
cd ../
pip install pandas psutil scipy pyyaml
## tim_models
cd pytorch
TIMM_COMMIT_ID="$(cat .ci/docker/ci_commit_pins/timm.txt 2> /dev/null || echo 'v1.0.14')"
pip install git+https://github.com/huggingface/pytorch-image-models@${TIMM_COMMIT_ID}
## torchbench
cd pytorch
TORCHBENCH_COMMIT_ID="$(cat .ci/docker/ci_commit_pins/torchbench.txt 2> /dev/null || echo 'e03a63be')"
cd ../
rm -rf ./benchmark
git clone https://github.com/pytorch/benchmark benchmark
cd benchmark
git checkout ${TORCHBENCH_COMMIT_ID}
pip install -r requirements.txt
python install.py --continue_on_fail phlippe_resnet functorch_dp_cifar10
pip install pyre-extensions
curl -fsSL https://raw.githubusercontent.com/facebookresearch/dlrm/refs/heads/torchrec-dlrm/requirements.txt |xargs pip install
pip uninstall -y fbgemm_gpu_nightly-cpu
pip install git+https://github.com/nocoding03/gym@fix-np

pip install torchao --index-url https://download.pytorch.org/whl/test/xpu
pip install -U numpy==1.26.4
## install target install
pip uninstall torch -y
pip install --no-deps /download/path/torch-*.whl
##run command line
python benchmarks/dynamo/timm_models.py --performance -d xpu -n10 --backend=inductor --cold-start-latency --inference --float32 --only tnt_s_patch16_224
python benchmarks/dynamo/torchbench.py --performance -d xpu -n10 --backend=inductor --cold-start-latency --training --amp --amp-dtype bfloat16 --only phlippe_resnet
python benchmarks/dynamo/torchbench.py --performance -d xpu -n10 --backend=inductor --cold-start-latency --training --float16 --only functorch_dp_cifar10
```

### Versions

torch 2.14.0.dev20260901+xpu
torchao 0.18.0+xpu
torchaudio 2.11.0+xpu
torchvision 0.29.0+xpu

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.