Azure / Azure/AzurePublicDataset
Missing prefix cache hit rate in Azure LLM inference trace 2023 and 2024.
Open
- Dominant language
- Jupyter Notebook
- Stars
- 1.2k
- Forks
- 185
- PR merge metrics
- No merged PRs in 30d
Description
In these two traces, there are only input and output lengths, but no prefix cache hit rate. Therefore, the data tested using these traces cannot truly reflect the inference performance and load conditions. Could you please supplement these traces with their corresponding prefix cache hit rates?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.