mlcommons / mlcommons/modelbench
Investigate proxy for clients to provision a model
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 134
- Forks
- 36
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 17
Description
Currently clients need to handle resource creation themselves (e.g. "give me this huggingface resource so I can run my test." This means they have to know the implementation details of the resource they need and we can't easily manage resource allocation, queuing, turning resources off when not needed, etc.
Investigate a proxy that would handle those things for clients, so they can just hit the proxy with a unified interface, and the proxy would handle the details.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points; start by locating the current client-side resource creation and allocation flow. Define the proxy's unified interface and lifecycle requirements, with completion being an agreed implementation plan or prototype that covers queuing, allocation, and shutting resources down.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100