[OOB][Perf] aten::native_batch_norm performance gap on B580 (13 models affected)
- Dominant language
- Python
- Stars
- 113
- Forks
- 129
- Avg merge
- 5d 9h
- Merged PRs (30d)
- 112
Description
## Summary
`aten::native_batch_norm` is the top 4 op priority on B580 for XPU performance improvement in OOB 300 eager inference (fp16). The gap is significantly larger on B580 than G31, suggesting a platform-specific kernel issue.
## Data (from roofline profiling, 154 models, fp16 inference)
| Platform | Geomean R Delta | Models Affected | Total Saving (ms) |
|----------|------:|------:|------:|
| B580 | +0.0049 | 13 | 6,168 |
| G31 (B70) | +0.0012 | 10 | 145 |
Current fleet geomean R: B580=0.644, G31=0.675, 4080S=0.683.
## Note
The B580 total saving (6,168 ms) is 42x larger than G31 (145 ms) despite fewer affected models. This strongly suggests a B580-specific performance issue with the batch_norm kernel — likely related to occupancy, vectorization, or memory access patterns on BMG architecture.
Batch norm is a memory-bound op (vector engine). The roofline projects it at bytes/BW, so a gap means the kernel is not achieving peak bandwidth on B580.
## Hardware
- B580: 93 TFLOPS fp16, 410 GB/s
- G31 (B70): 154 TFLOPS fp16, 532 GB/s
- 4080S: 100.96 TFLOPS fp16, 716.8 GB/s
Contributor guide
Research direction
Start at the aten::native_batch_norm entry point and reproduce the fp16 eager-inference measurements on B580, comparing them with G31. Use the roofline data and the 13 affected models to determine whether the gap is reproducible; done means a measured B580 performance improvement with updated results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pytorch
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100