用lmdeploy部署internlm2_5-7B-chat请求返回为空
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
### 📚 The doc issue
### Code部分:
```
def generate_QA():
server_addr = "..."
client = APIClient(server_addr, api_key=None)
model_name = client.available_models[0]
req = "你是一位善于总结文本大纲的机器人,你的任务是按照以下步骤分析长文本内容,并按照要求进行输出......"
prompt = [{"role": "user", "content": req}]
top_p = 0.9
temperature = 0.7
output_seqlen = 4096
stream_output = False
for response in client.chat_completions_v1(model=model_name, messages=prompt, temperature=temperature, top_p=top_p, n=1, max_tokens=output_seqlen, stream=stream_output, ignore_eos=False):
print(response)
output = response["choices"][0]['message']['content']
return output
print(generate_QA())
```
如果我的prompt是比较简单的问题。比如“介绍一下自己”,返回结果就不为空,但如果prompt较为复杂,返回结果就是空值。如下图:


### Suggest a potential alternative/fix
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.