[Driver 26.19] Performance drop for torchbench moondream amp_fp16 inference
- Dominant language
- Python
- Stars
- 113
- Forks
- 129
- Avg merge
- 5d 9h
- Merged PRs (30d)
- 112
Description
## Summary
Driver 26.19 shows a confirmed performance drop for one torchbench case compared with driver 25.40.
## Affected case
| Suite | Dtype | Mode | Model | Eager Latency ratio |
| --- | --- | --- | --- | --- |
| torchbench | amp_fp16 | inference | moondream | 0.931077727 |
## Reproducer
### Environment setup
```bash
docker run -it -e TZ=Aisa/Shanghai --device=/dev/mem --device=/dev/dri --group-add video --privileged --shm-size=8g intelgpu/ubuntu-24.04-lts2:2523.40 bash
curl -LsSf https://astral.sh/uv/install.sh | env UV_INSTALL_DIR="/usr/local/bin" sh
uv venv myvenv --python 3.12 --clear
source myvenv/bin/activate
uv pip install pip setuptools wheel
uv pip install torch torchaudio torchvision --index-url https://download.pytorch.org/whl/xpu
pytorch_commit="$(python -c 'import torch; print(torch.version.git_version)')"
git clone https://github.com/pytorch/pytorch
cd pytorch
git checkout ${pytorch_commit}
uv pip install pip pandas psutil scipy requests
uv pip install -U numpy==1.26.4 accelerate
uv pip install -r .ci/docker/ci_commit_pins/huggingface-requirements.txt
uv pip install timm
rm -rf torch torch-xpu-ops
git clone https://github.com/intel/torch-xpu-ops
rsync -avz ./torch-xpu-ops/.ci/benchmarks/ ./benchmarks/dynamo/
git clone https://github.com/pytorch/benchmark tmp-benchmark
cd tmp-benchmark
python install.py --continue_on_fail
export PYTHONPATH=$PWD
cd ..
# update torch
# gh --repo intel/torch-xpu-ops run download 26399638675 -p "Torch-XPU-Wheel-*"
uv pip uninstall torch torchaudio torchvision triton-xpu -y
uv pip install Torch-XPU-Wheel-*/*.whl
```
### Performance command
```bash
python benchmarks/dynamo/torchbench.py -d xpu --performance --inference --amp --amp-dtype float16 --backend=inductor --cold-start-latency -n10 --disable-cudagraphs --only moondream --output=/tmp/tmp.csv
cat /tmp/tmp.csv # eager latency = abs_latency * speedup
```
## Source runs
- Target: https://github.com/intel/torch-xpu-ops/actions/runs/26264076598
- Baseline: https://github.com/intel/torch-xpu-ops/actions/runs/26267132121
## Notes
Latency ratio is computed as baseline / target. Values below 1 indicate target is slower.
Contributor guide
Research direction
Start by running the provided benchmarks/dynamo/torchbench.py command for moondream with the target and baseline driver runs, then compare the generated latency results. The issue is complete when the regression is explained and an appropriate correction or confirmed upstream cause is documented, with the performance comparison reproduced.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100