[Issue]: Attention not correct
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 565
- Forks
- 585
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 366
Description
Problem Description
We found the attention OP with
(seqlen_q=1024, seqlen_kv=1024, num_head_q=4, num_head_kv=4, head_dim_qk=128, head_dim_v=128)
(seqlen_q=1024, seqlen_kv=1024, num_head_q=12, num_head_kv=4, head_dim_qk=128, head_dim_v=128)
will core dump or not correct after we update aiter from
(-Subproject commit 568410e0a2bfa7e7f9878b0215000f0c871f5686)
to
(+Subproject commit 5e9928fab19d5e3cae4b5bebb4dea4ec6e9fecad)
Operating System
NAME="Ubuntu" VERSION="24.04.1 LTS (Noble Numbat)"
CPU
AMD EPYC 9654 96-Core Processor
GPU
AMD Instinct MI300X * 8
ROCm Version
6.10.5
ROCm Component
No response
Steps to Reproduce
contact Cheng.Yao2@amd.com
(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support
No response
Additional Information
No response
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the attention OP configurations on Ubuntu 24.04.1 with ROCm 6.10.5 and eight MI300X GPUs, comparing the two listed aiter subproject commits. The issue provides no file or test entry point; done means both configurations complete without a core dump and produce correct results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100