bigscience-workshop / bigscience-workshop/petals
Report average queue size in tokens (per last 10 min) for routing
Open
- Dominant language
- Python
- Stars
- 10.6k
- Forks
- 650
- PR merge metrics
- No merged PRs in 30d
Description
This would help to take the server load into account while planning a route for inference and fine-tuning.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.