aws-samples / aws-samples/foundation-model-benchmarking-tool
Support speculative decoding
Open
- Dominant language
- Jupyter Notebook
- Stars
- 256
- Forks
- 44
- PR merge metrics
- No merged PRs in 30d
Description
Is it on the roadmap for this tool to support inference optimization such as speculative decoding for gpus and neuron cores?
Contributor guide
Assessment
This issue has not been assessed yet.