vllm-project / vllm-project/aibrix
Implement model architect aware scheduling policies
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 694
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 98
Description
### 🚀 Feature Description and Motivation
Currently, runtime picks up the work to download the model weights. If we have another replica wants to be deployed, one option is to be scheduled to the same node already has weights. In this case, it's would be great that we can contribute some scheduler plugin to be aware of the artifacts.
### Use Case
_No response_
### Proposed Solution
_No response_
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by surveying the repository's runtime and scheduler-plugin architecture, then clarify how model artifacts should influence replica placement; done requires an agreed design and an implementation with validation for artifact-aware scheduling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100