microsoft / microsoft/MInference
[Question]: what is the speedup of attention kernel of current implemetation?
Open
@iofu728 is already working on this.
Since Sep 11, 2024.
question
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 82
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 1
Description
Describe the issue
The pattern is good, but I wonder if we have hardware efficient kernel of these patterns. Have you test this sparse SDPA attention kernel speedup compared to original causal attention?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.