microsoft / microsoft/onnxruntime-genai
[feature request] builder to expose {GQA, MHA} selection as argument
Open
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
Currently these are inferred from the combination of other configurations such as device and dtype. It is more flexible for downstream users if this can be selected by choice.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.