microsoft / microsoft/onnxruntime
Handle position ids and attention bias inputs in GQA for non-CPU EPs
Open
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
In https://github.com/microsoft/onnxruntime/pull/23944, support for position ids and attention bias optional inputs was introduced for GQA CPU operator. These inputs are not supported yet for other EPs so add checks if these inputs are used in op implementation for other EPs and throw not implemented exception
cc @tianleiwu
Contributor guide
Assessment
This issue has not been assessed yet.