[Inductor] kernel impl and perf analysis for torch/_inductor/kernel
- Dominant language
- Python
- Stars
- 113
- Forks
- 128
- Avg merge
- 5d 9h
- Merged PRs (30d)
- 112
Description
### 🚀 The feature, motivation and pitch
# XPU support for kernels in `torch/_inductor/kernel` Task List
**Goal:** Implement missing Triton template kernels for XPU under torch/_inductor/kernel, analyze and understand the performance efficiency and its bottlenecks, and explore alternative backends (e.g. SYCL-TLA) to achieve better performance if Triton is unable to deliver optimal results.
**Please add your name to the Assign column, and append your results — including PRs, performance analysis (BMG), and related findings in the comment of this thread context.**
| Fusion Pattern | Category | Source | Triton Template | SYCL-TLA template | Assign |
|----------------|----------|--------|-----------------|-------------------|--------|
| `aten._int_mm` (INT8 GEMM) | Inductor GEMM | [torch/_inductor/kernel/mm.py · `tuned_int_mm`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/mm.py) | Y | N | Xiao, Wang|
| `aten._scaled_mm` (FP8 scaled MM) | Inductor GEMM+Quant | [torch/_inductor/kernel/mm.py · `tuned_scaled_mm`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/mm.py) | Y | N| Xiao, Wang|
| `aten._scaled_mm_v2` (FP8 multi-level scaling) | Inductor GEMM+Quant | [torch/_inductor/kernel/mm.py · `tuned_scaled_mm_v2`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/mm.py) | Y | N |Xiao, Wang|
| `aten._sparse_semi_structured_mm` (2:4 sparse MM) | Inductor GEMM | [torch/_inductor/kernel/mm.py · `tuned_sparse_semi_structured_mm`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/mm.py) | Y | N | Han, Xu |
| `aten._grouped_mm` (grouped / MoE GEMM) | Inductor MoE | [torch/_inductor/kernel/mm_grouped.py · `tuned_grouped_mm`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/mm_grouped.py) | Y | N | Lin, Xinan |
| `aten._scaled_grouped_mm` (scaled grouped / MoE GEMM) | Inductor MoE | [torch/_inductor/kernel/mm_grouped.py · `tuned_scaled_grouped_mm`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/mm_grouped.py) | Y | N | Chen, Long |
| `mm_plus_mm` (A@B + C@D fused) | Inductor GEMM | [torch/_inductor/kernel/mm_plus_mm.py · `tuned_mm_plus_mm`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/mm_plus_mm.py) | Y | N | Han, Xu |
| B2B GEMM (`A @ f(B @ C)` back-to-back) | Inductor GEMM | [torch/_inductor/fx_passes/b2b_gemm.py · `b2b_gemm_handler`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/fx_passes/b2b_gemm.py) | Y | N | Han, Xu |
| INT8/FP8 GEMM + dequant/requant quantization fusion | Inductor GEMM+Quant | [torch/_inductor/fx_passes/quantization.py](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/fx_passes/quantization.py) | Y | N | Xiao, Wang |
| Flex Attention (fused MHA + score_mod) | Inductor Attention | [torch/_inductor/kernel/flex/flex_attention.py](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/flex/flex_attention.py) | Y | N | Chunhuan Meng|
| Flex Decoding (decode-time fused attention) | Inductor Attention | [torch/_inductor/kernel/flex/flex_decoding.py](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/flex/flex_decoding.py) | Y | N | Chunhuan Meng|
### Alternatives
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.