InternLM / InternLM/lmdeploy

[Bug] lora微调后一直重复输出

Open
#2,757 2 comments 0 reactions 1 assignee Claimed by @lvhan028 View on GitHub
Dominant language
Python
Stars
8.1k
Forks
748
Avg merge
6d 2h
Merged PRs (30d)
54

Description

### Checklist

- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
- [X] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

### Describe the bug

使用swift微调后,在swift上运行一切正常,使用lmdeploy部署后,出现一直重复输出没有停止的情况
启动命令 :
lmdeploy serve api_server --backend pytorch --device ascend --eager-mode /workspace/data/swift/yinglin3/output/llama2-7b-chat/v2-20241114-061210/checkpoint-60-merged --tp 1 --server-port 23334 --model-name llama2

swift训练命令:
ASCEND_RT_VISIBLE_DEVICES=0 \
swift sft \
--model_type llama2-7b-chat \
--dataset self-cognition \
--num_train_epochs 5 \
--sft_type lora \
--output_dir output \
--model_name xxx \
--model_author xx

请求url:
curl -X POST http://0.0.0.0:23334/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "llama2",
"stream":false,
"messages": [
{
"role": "user",
"content": "你是谁"
}
]
}'

输出:非流式输出,流式同样是这样
{"id":"20","object":"chat.completion","created":1731574617,"model":"llama2","choices":[{"index":0,"message":{"role":"assistant","content":" I am a machine learning model developed by 魔搭, my name is 小黄。 I am designed to understand and respond to human language, and provide assistance and answer questions to the best of my ability.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n ","tool_calls":null},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":94,"total_tokens":666,"completion_tokens":572}}

### Reproduction

lmdeploy serve api_server --backend pytorch --device ascend --eager-mode /workspace/data/swift/yinglin3/output/llama2-7b-chat/v2-20241114-061210/checkpoint-60-merged --tp 1 --server-port 23334 --model-name llama2

### Environment

```Shell
华为昇腾910b,训练和推理都使用的是910b
```

### Error traceback

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.