InseeFrLab / InseeFrLab/auto-tuning-vllm
Implement a Fixed Schedule dataset from production traces distribution
Open
- Dominant language
- Python
- Stars
- 6
- Forks
- 0
- Avg merge
- 34m
- Merged PRs (30d)
- 2
Description
As a user, i would like to generate a dataset in this format :
`{"timestamp": 0, "input_length": 100, "output_length": 200, "hash_ids": [1001]}
{"timestamp": 500, "input_length": 200, "output_length": 400, "hash_ids": [1002]}
{"timestamp": 1000, "input_length": 550, "output_length": 500, "hash_ids": [1003, 1005]}`
using the distribution from ai gateway traces.
- [ ] Model the probability distribution of the AI gateway
- [ ] Test with AI perf https://github.com/ai-dynamo/aiperf
- [ ] Extract traces from litellm
- [ ] Extract traces from opengatellm
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.