[Bug] lora微调后一直重复输出
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
### Checklist
- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
- [X] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
### Describe the bug
使用swift微调后,在swift上运行一切正常,使用lmdeploy部署后,出现一直重复输出没有停止的情况
启动命令 :
lmdeploy serve api_server --backend pytorch --device ascend --eager-mode /workspace/data/swift/yinglin3/output/llama2-7b-chat/v2-20241114-061210/checkpoint-60-merged --tp 1 --server-port 23334 --model-name llama2
swift训练命令:
ASCEND_RT_VISIBLE_DEVICES=0 \
swift sft \
--model_type llama2-7b-chat \
--dataset self-cognition \
--num_train_epochs 5 \
--sft_type lora \
--output_dir output \
--model_name xxx \
--model_author xx
请求url:
curl -X POST http://0.0.0.0:23334/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "llama2",
"stream":false,
"messages": [
{
"role": "user",
"content": "你是谁"
}
]
}'
输出:非流式输出,流式同样是这样
{"id":"20","object":"chat.completion","created":1731574617,"model":"llama2","choices":[{"index":0,"message":{"role":"assistant","content":" I am a machine learning model developed by 魔搭, my name is 小黄。 I am designed to understand and respond to human language, and provide assistance and answer questions to the best of my ability.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n ","tool_calls":null},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":94,"total_tokens":666,"completion_tokens":572}}
### Reproduction
lmdeploy serve api_server --backend pytorch --device ascend --eager-mode /workspace/data/swift/yinglin3/output/llama2-7b-chat/v2-20241114-061210/checkpoint-60-merged --tp 1 --server-port 23334 --model-name llama2
### Environment
```Shell
华为昇腾910b,训练和推理都使用的是910b
```
### Error traceback
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.