huggingface / huggingface/optimum-amd
TGI Improvements for ROCM 6.2
Open
Instinct
- Dominant language
- Jupyter Notebook
- Stars
- 101
- Forks
- 21
- Avg merge
- 58m
- Merged PRs (30d)
- 1
Description
- [x] Update CK flash attention to use CK tile
- [x] Update VLLM to latest rocm/vllm commit.
- [x] Update torch
- [x] Fix tunable op issue with TP8
- [x] BF16 inference fix
- [x] Custom Paged attention
- [x] Documentation
- [ ] Rebase
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.