[MLIR][Attention] Implement gemm(i8)-dequantizelinear-softmax(fp16)-gemm(fp16) lowering
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 333
- Forks
- 150
- Avg merge
- 4d 19h
- Merged PRs (30d)
- 54
Description
Problem Description
This ticket is to implement gemm(i8)-dequantizelinear-softmax(fp16)-gemm(fp16) pattern to do a partial i8 attention kernel in rocmlir.
Here is one of the examples test we currently have working : https://github.com/ROCm/rocMLIR/blob/develop/mlir/test/fusion/pr-e2e/attention/mixr-attention-first-gemm-i8-f16.mlir
Operating System
Any
CPU
Any
GPU
AMD Instinct MI300X, AMD Instinct MI250X, AMD Instinct MI250, AMD Instinct MI210
Other
No response
ROCm Version
ROCm 6.0.0
Steps to Reproduce
No response
(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support
No response
Additional Information
No response
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the working example in mlir/test/fusion/pr-e2e/attention/mixr-attention-first-gemm-i8-f16.mlir and use it to understand the gemm(i8)-dequantizelinear-softmax(fp16)-gemm(fp16) pattern. Trace the relevant MLIR lowering entry points, then validate that the requested pattern lowers successfully for the listed attention case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100