InternLM / InternLM/lmdeploy

是否支持多模态的多lora推理

Open
#4,067 2 comments 0 reactions 1 assignee Claimed by @grimoire View on GitHub
Dominant language
Python
Stars
8.1k
Forks
748
Avg merge
6d 2h
Merged PRs (30d)
54

Description

### Motivation

对不同领域都微调了一个lora adapter(基于qwen2.5-vl-7b),这个时候就希望,在显存不猛增的情况下,灵活地动态更改模型,也就是需要推理框架支持多lora推理,可以切换 adapter,**当前我们的服务是否支持,或者未来会支持吗**

### Related resources

vllm有提供这个Image
https://docs.vllm.ai/en/latest/features/lora.html#serving-lora-adapters

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.