aws / aws/amazon-q-developer-cli
feat: Display tokens per second
Open
Feature request
q chat
- Dominant language
- Rust
- Stars
- 2k
- Forks
- 439
- PR merge metrics
- No merged PRs in 30d
Description
Show the tokens-per-second (and possibly more detailed information like eval rate, durations, etc) for each message during a conversation. Could show this when Q is run with `--verbose`
Inspiration from https://www.reddit.com/r/LocalLLaMA/comments/1egxdpt/post_your_tokens_per_second_for_llama3170b/
Contributor guide
Assessment
This issue has not been assessed yet.