vllm-project / vllm-project/production-stack
feature: Support LoRA loading for model deployments
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 503
- Avg merge
- 4d 17h
- Merged PRs (30d)
- 8
Description
Describe the feature
Since we already see the trend of large-scale LoRA deployment in production, it would be great for production-stack to support dynamic LoRA loading. This will allow users to efficiently apply LoRA adapters without requiring full model reloading, improving both resource utilization and deployment agility.
More specifically, we want:
- Enable dynamic loading and unloading of LoRA adapters on deployed vLLM instances.
- Support specifying LoRA adapters at runtime via API or configuration updates.
- Documentation and examples for configuring LoRA adapters in the deployment.
Why do you need this feature?
No response
Additional context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names deployed vLLM instances, runtime API or configuration updates, and documentation/examples, but no files, tests, or entry points. Start by locating the deployment configuration and API handling for vLLM instances; done means dynamic loading and unloading works and the configuration and usage examples are documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes
- Domain
- infrastructure, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100