InternLM / InternLM/lmdeploy

[Bug] 无法部署https://github.com/PKU-Alignment/safe-rlhf这里的大模型

Open
#1,460 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
8.1k
Forks
748
Avg merge
6d 2h
Merged PRs (30d)
54

Description

### Checklist

- [ ] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.

### Describe the bug

# pipe = lmdeploy.pipeline("/home/tcexeexe/mode_deploy/tcexeexe/SHTEC_RM_model")

# response = pipe(["BEGINNING OF CONVERSATION: USER: 你是个傻逼吗? ASSISTANT: 是的。"])

# print(response)

# """
# AssertionError: 'None' is not supported. The supported models are: dict_keys(['base', 'llama', 'internlm', 'vicuna', 'wizardlm', 'internlm-chat-7b', 'internlm-chat', 'internlm-chat-7b-8k', 'internlm-chat-20b', 'internlm-20b', 'internlm2-7b', 'internlm2-20b', 'internlm2-chat-7b', 'internlm2-chat-20b', 'baichuan-7b', 'baichuan2-7b', 'puyu', 'llama2', 'llama-2', 'llama-2-chat', 'qwen-7b', 'qwen-14b', 'codellama', 'falcon', 'chatglm2-6b', 'solar', 'solar-70b', 'ultralm', 'ultracm', 'yi', 'yi-chat', 'yi-200k', 'yi-34b'])

### Reproduction

from modelscope.hub.api import HubApi
api = HubApi()
api.login()

# 模型下载
# from modelscope import snapshot_download
# model_dir = snapshot_download('tcexeexe/SHTEC_RM_model',cache_dir='/home/tcexeexe/mode_deploy')

# /home/tcexeexe/mode_deploy/tcexeexe/SHTEC_RM_model

import lmdeploy

# pipe = lmdeploy.pipeline("/home/tcexeexe/mode_deploy/tcexeexe/SHTEC_RM_model")

# response = pipe(["BEGINNING OF CONVERSATION: USER: 你是个傻逼吗? ASSISTANT: 是的。"])

# print(response)

# """
# AssertionError: 'None' is not supported. The supported models are: dict_keys(['base', 'llama', 'internlm', 'vicuna', 'wizardlm', 'internlm-chat-7b', 'internlm-chat', 'internlm-chat-7b-8k', 'internlm-chat-20b', 'internlm-20b', 'internlm2-7b', 'internlm2-20b', 'internlm2-chat-7b', 'internlm2-chat-20b', 'baichuan-7b', 'baichuan2-7b', 'puyu', 'llama2', 'llama-2', 'llama-2-chat', 'qwen-7b', 'qwen-14b', 'codellama', 'falcon', 'chatglm2-6b', 'solar', 'solar-70b', 'ultralm', 'ultracm', 'yi', 'yi-chat', 'yi-200k', 'yi-34b'])

### Environment

```Shell
正常的cuda环境
```

### Error traceback

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.