InternLM / InternLM/lmdeploy

[Feature] Is there any plan to support for InternLM-XComposer2.5 inference?

Open
#1,920 2 comments 0 reactions 1 assignee Claimed by @irexyc View on GitHub
Dominant language
Python
Stars
8.1k
Forks
748
Avg merge
6d 2h
Merged PRs (30d)
54

Description

### Motivation

InternLM-Xcomposer2.5 just got released and its performance is really amazing.

I try to use lmdeploy to accelerate the inference, but failed. Is there any plan to support it?

I would appreciate it very much, and I think it can benefit the community a lot, too.

### Related resources

image

I tried to run xcomposer2.5 with lmdeploy, but I came across this problem. I guess lmdeploy does not support xcomposer2.5 inference now.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.