intel / intel/torch-xpu-ops

[Driver 26.19] Performance drop for torchbench moondream amp_fp16 inference

Open
#3,790 3 comments 0 reactions 0 assignees View on GitHub
dependency component: driver hw: PVC
Dominant language
Python
Stars
113
Forks
129
Avg merge
5d 9h
Merged PRs (30d)
112

Description

## Summary
Driver 26.19 shows a confirmed performance drop for one torchbench case compared with driver 25.40.

## Affected case
| Suite | Dtype | Mode | Model | Eager Latency ratio |
| --- | --- | --- | --- | --- |
| torchbench | amp_fp16 | inference | moondream | 0.931077727 |

## Reproducer
### Environment setup
```bash
docker run -it -e TZ=Aisa/Shanghai --device=/dev/mem --device=/dev/dri --group-add video --privileged --shm-size=8g intelgpu/ubuntu-24.04-lts2:2523.40 bash

curl -LsSf https://astral.sh/uv/install.sh | env UV_INSTALL_DIR="/usr/local/bin" sh
uv venv myvenv --python 3.12 --clear
source myvenv/bin/activate
uv pip install pip setuptools wheel
uv pip install torch torchaudio torchvision --index-url https://download.pytorch.org/whl/xpu

pytorch_commit="$(python -c 'import torch; print(torch.version.git_version)')"
git clone https://github.com/pytorch/pytorch
cd pytorch
git checkout ${pytorch_commit}

uv pip install pip pandas psutil scipy requests
uv pip install -U numpy==1.26.4 accelerate
uv pip install -r .ci/docker/ci_commit_pins/huggingface-requirements.txt
uv pip install timm

rm -rf torch torch-xpu-ops
git clone https://github.com/intel/torch-xpu-ops
rsync -avz ./torch-xpu-ops/.ci/benchmarks/ ./benchmarks/dynamo/

git clone https://github.com/pytorch/benchmark tmp-benchmark
cd tmp-benchmark
python install.py --continue_on_fail
export PYTHONPATH=$PWD
cd ..

# update torch
# gh --repo intel/torch-xpu-ops run download 26399638675 -p "Torch-XPU-Wheel-*"
uv pip uninstall torch torchaudio torchvision triton-xpu -y
uv pip install Torch-XPU-Wheel-*/*.whl
```

### Performance command
```bash
python benchmarks/dynamo/torchbench.py -d xpu --performance --inference --amp --amp-dtype float16 --backend=inductor --cold-start-latency -n10 --disable-cudagraphs --only moondream --output=/tmp/tmp.csv
cat /tmp/tmp.csv # eager latency = abs_latency * speedup
```

## Source runs
- Target: https://github.com/intel/torch-xpu-ops/actions/runs/26264076598
- Baseline: https://github.com/intel/torch-xpu-ops/actions/runs/26267132121

## Notes
Latency ratio is computed as baseline / target. Values below 1 indicate target is slower.

Contributor guide

Open the contributing guide

Research direction

Start by running the provided benchmarks/dynamo/torchbench.py command for moondream with the target and baseline driver runs, then compare the generated latency results. The issue is complete when the regression is explained and an appropriate correction or confirmed upstream cause is documented, with the performance comparison reproduced.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python, pytorch
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.