EmbarkStudios / EmbarkStudios/cervo
serve: add loadtest subcommand for use with `cervo serve` or other hosting.
Open
enhancement
- Dominant language
- Rust
- Stars
- 56
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Would be interesting to see how far we can scale cervo serve or other inf servers perf-wise, to avoid needing GPU for distributed serving of small models.
Contributor guide
Assessment
This issue has not been assessed yet.