huggingface / huggingface/HuggingSnap
Prompt Time
Open
- Dominant language
- Swift
- Stars
- 189
- Forks
- 20
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I just want to ask a question about the prompt time.
When I try to get a response from the 500M model, it takes approximately 5 seconds to get the initial token (Used iPhone 13).
I just want to know why this is and there any way or plans to get this time faster?
https://github.com/huggingface/HuggingSnap/blob/88aafba3354775ffac174bc44535f8908b8a679a/HuggingSnap/Views/VLMEvaluator.swift#L195
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.