Paged decode failure cases on MI350X
Open
- Dominant language
- Python
- Stars
- 59
- Forks
- 32
- PR merge metrics
- No merged PRs in 30d
Description
The following configurations result in numerical inaccuracies when running paged decode attention test on MI350X.
```
(NUM_Q_HEADS, NUM_KV_HEADS, HEAD_SIZE, HEAD_SIZE_KV, BLOCK_SIZE, NUM_SEQS, SEQ_LEN)
shapes = [(16, 16, 64, 64, 32, 2, 5)] *
shapes += [(16, 1, 64, 64, 32, 2, 5)]
shapes += [(128, 1, 80, 80, 32, 2, 5)]
shapes += [(128, 2, 512, 512, 32, 32, 5)] *
shapes += [(128, 2, 512, 512, 32, 2, 5)]
shapes += [(128, 2, 576, 512, 32, 2, 5)] *
```
/* - fails on MI300X as well
Contributor guide
Assessment
This issue has not been assessed yet.