Prompt Caching Expiration shown in Session/Chat
Open
- Dominant language
- TypeScript
- Stars
- 193k
- Forks
- 42.4k
- PR merge metrics
- PR metrics pending
Description
Since parallel running Sessions and Chats is very common and Costs have risen, managing Tokens and therefore Cache hits is very important.
It would be very useful to know for every response how long this is cached.
- When is the cache lost
- Show this on Session info in the sessions list
- show this in the Session
This would allow taking care which session to prioritize.
Contributor guide
Assessment
This issue has not been assessed yet.