intel / intel/torch-xpu-ops

[Inductor] kernel impl and perf analysis for torch/_inductor/kernel

Open
#3,863 0 comments 0 reactions 3 assignees Claimed by @chunhuanMeng View on GitHub
Dominant language
Python
Stars
113
Forks
128
Avg merge
5d 9h
Merged PRs (30d)
112

Description

### 🚀 The feature, motivation and pitch

# XPU support for kernels in `torch/_inductor/kernel` Task List

**Goal:** Implement missing Triton template kernels for XPU under torch/_inductor/kernel, analyze and understand the performance efficiency and its bottlenecks, and explore alternative backends (e.g. SYCL-TLA) to achieve better performance if Triton is unable to deliver optimal results.

**Please add your name to the Assign column, and append your results — including PRs, performance analysis (BMG), and related findings in the comment of this thread context.**

| Fusion Pattern | Category | Source | Triton Template | SYCL-TLA template | Assign |
|----------------|----------|--------|-----------------|-------------------|--------|
| `aten._int_mm` (INT8 GEMM) | Inductor GEMM | [torch/_inductor/kernel/mm.py · `tuned_int_mm`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/mm.py) | Y | N | Xiao, Wang|
| `aten._scaled_mm` (FP8 scaled MM) | Inductor GEMM+Quant | [torch/_inductor/kernel/mm.py · `tuned_scaled_mm`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/mm.py) | Y | N| Xiao, Wang|
| `aten._scaled_mm_v2` (FP8 multi-level scaling) | Inductor GEMM+Quant | [torch/_inductor/kernel/mm.py · `tuned_scaled_mm_v2`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/mm.py) | Y | N |Xiao, Wang|
| `aten._sparse_semi_structured_mm` (2:4 sparse MM) | Inductor GEMM | [torch/_inductor/kernel/mm.py · `tuned_sparse_semi_structured_mm`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/mm.py) | Y | N | Han, Xu |
| `aten._grouped_mm` (grouped / MoE GEMM) | Inductor MoE | [torch/_inductor/kernel/mm_grouped.py · `tuned_grouped_mm`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/mm_grouped.py) | Y | N | Lin, Xinan |
| `aten._scaled_grouped_mm` (scaled grouped / MoE GEMM) | Inductor MoE | [torch/_inductor/kernel/mm_grouped.py · `tuned_scaled_grouped_mm`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/mm_grouped.py) | Y | N | Chen, Long |
| `mm_plus_mm` (A@B + C@D fused) | Inductor GEMM | [torch/_inductor/kernel/mm_plus_mm.py · `tuned_mm_plus_mm`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/mm_plus_mm.py) | Y | N | Han, Xu |
| B2B GEMM (`A @ f(B @ C)` back-to-back) | Inductor GEMM | [torch/_inductor/fx_passes/b2b_gemm.py · `b2b_gemm_handler`](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/fx_passes/b2b_gemm.py) | Y | N | Han, Xu |
| INT8/FP8 GEMM + dequant/requant quantization fusion | Inductor GEMM+Quant | [torch/_inductor/fx_passes/quantization.py](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/fx_passes/quantization.py) | Y | N | Xiao, Wang |
| Flex Attention (fused MHA + score_mod) | Inductor Attention | [torch/_inductor/kernel/flex/flex_attention.py](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/flex/flex_attention.py) | Y | N | Chunhuan Meng|
| Flex Decoding (decode-time fused attention) | Inductor Attention | [torch/_inductor/kernel/flex/flex_decoding.py](https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/flex/flex_decoding.py) | Y | N | Chunhuan Meng|

### Alternatives

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.