是否支持多模态的多lora推理
Open
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
### Motivation
对不同领域都微调了一个lora adapter(基于qwen2.5-vl-7b),这个时候就希望,在显存不猛增的情况下,灵活地动态更改模型,也就是需要推理框架支持多lora推理,可以切换 adapter,**当前我们的服务是否支持,或者未来会支持吗**
### Related resources
vllm有提供这个
https://docs.vllm.ai/en/latest/features/lora.html#serving-lora-adapters
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.