codellama-Instruct版本地测试运行和服务部署返回的结果不一致

### Checklist

- [x] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.

### Describe the bug

我本地运行codellama，模型用的7b-Instruct版，代码如下：

```python
dialogs1: List[Dialog] = [
[{"role": "system",
"content": SystemPrompt1},
{"role": "user",
"content": UserPrompt1.format(icall_context[-1], "\n".join(icall_context),
"\n\n".join(function_declarators))}]
]
results1: List[ChatPrediction] = model.chat_completion(
dialogs1, # type: ignore
)
output1: ChatPrediction = results1[0]
content1 = output1['generation']['content']
```

内容为

```
[
{'role': 'system', 'content': 'You are a code analyzer tasked with evaluating the likelihood of an indirect callsite effectively invoking certain functions. \nYour analysis should primarily focus on assessing the semantic similarity between the indirect call, its context, and the function declarators. \nAdditionally, consider the correspondence between formal and actual parameter names, as well as the compatibility of type names. \nYour goal is to assess the likelihood of the indirect callsite invoking these functions and provide a reasoned prediction.\n'},

{'role': 'user', 'content': "Please evaluate the following indirect call:\n\np = log->handler(log, p, last - p);\n\nThe context provides information about variables and types, as follows:\n\nngx_log_t *log\nu_char *p, *last, *msg;\np = log->handler(log, p, last - p);\n\nTheir function declarators are as follows:\n\nchar * \nngx_conf_set_sec_slot(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)\n\nstatic char *\nngx_stream_ssl_preread_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child)\n\nstatic u_char *\nngx_resolver_log_error(ngx_log_t *log, u_char *buf, size_t len)\n\nu_char *\nngx_accept_log_error(ngx_log_t *log, u_char *buf, size_t len)\n\nstatic char *\nngx_http_fastcgi_lowat_check(ngx_conf_t *cf, void *post, void *data)\n\nstatic ngx_int_t\nngx_http_upstream_copy_content_encoding(ngx_http_request_t *r,\nngx_table_elt_t *h, ngx_uint_t offset)\n\nstatic ngx_int_t\nngx_stream_variable_proxy_protocol_port(ngx_stream_session_t *s,\nngx_stream_variable_value_t *v, uintptr_t data)\n\nYour analysis should determine if there is a substantial possibility of the indirect call effectively invoking any of the listed functions. \nPay special attention to semantic similarity between the indirect call, context, and function declarators, as well as the alignment of parameter names and type compatibility. \nProvide your assessment using the labels 'Yes' (for highly likely), 'Uncertain' (for not sure), or 'No' (for highly unlikely).\n"}
]
```

本地按照codellama官方示例运行的结果为：

```
Based on the provided information, the indirect call `log->writer(log, level, errstr, p - errstr);` is likely to effectively invoke the `ngx_syslog_writer` function.

The `log->writer` field is of type `ngx_log_writer_pt`, which is a pointer to a function that takes four arguments: a pointer to a `ngx_log_t` struct, an `ngx_uint_t` level, a `u_char` pointer to a buffer, and a `size_t` length. The `ngx_syslog_writer` function has the same signature, making it a likely candidate for the indirect call.

Furthermore, the context provides information about the `level`, `log`, `p`, `last`, `msg`, and `errstr` variables, which are all used in the `ngx_syslog_writer`
result2 is:
Based on the provided information, the indirect call `log->writer(log, level, errstr, p - errstr);` is likely to effectively invoke the `ngx_syslog_writer` function.
```

但是服务部署后用下面代码请求:

```python
headers = {
"Content-Type": "application/json"
}

prompt = [
{"role": "system", "content": SystemPrompt1},
{"role": "user", "content": UserPrompt1.format(icall_context[-1], "\n".join(icall_context),
"\n\n".join(func_name2declarator.values()))}
]

data = {
"model": "codellama",
"messages": prompt
}
# 定义server端的URL
server_url = "http://127.0.0.1:8888/v1/chat/completions" # 替换为实际的服务器地址和端口

# 发送POST请求，将JSON数据发送到server
response = requests.post(server_url, headers=headers, data=json.dumps(data))
```

结果变成了:

```
{'id': '127001', 'object': 'chat.completion', 'created': 1697613792, 'model': 'codellama', 'choices': [
{'index': 0,
'message':
{'role': 'assistant', 'content': 'Based on the information provided, there is a substantial possibility of the indirect callsite invoking the listed functions.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n'}, 'finish_reason': None}],
'usage': {'prompt_tokens': 609, 'total_tokens': 1122, 'completion_tokens': 513}}
```

测试了几轮，返回值都是这样，感觉后者比前者不靠谱有点多，是发送prompt的问题吗？

### Reproduction

本地运行的脚本为：

```python
from testcases.testcase1 import icallsite_context as icall_context1, \
function_declarators as function_declarators1, \
function_names as function_names1
from testcases.testcase2 import icallsite_context as icall_context2, \
function_declarators as function_declarators2, \
function_names as function_names2
from testcases.testcase3 import icallsite_context as icall_context3, \
function_declarators as function_declarators3, \
function_names as function_names3
from testcases.testcase4 import icallsite_context as icall_context4, \
function_declarators as function_declarators4, \
function_names as function_names4
from testcases.testcase5 import icallsite_context as icall_context5, \
function_declarators as function_declarators5, \
function_names as function_names5

import argparse
from codellama.generation import Llama, ChatPrediction, Dialog
from typing import List
from model_config import cache_dir

# 5个测试用例
icall_contexts: List[List[str]] = [icall_context1, icall_context2, icall_context3,
icall_context4, icall_context5]
function_names: List[List[str]] = [function_names1, function_names2, function_names3,
function_names4, function_names5]
function_declarators: List[List[str]] = [function_declarators1, function_declarators2,
function_declarators3, function_declarators4, function_declarators5]

def main():
parser = argparse.ArgumentParser(description="ICall Test")
parser.add_argument('--testcase', type=int, required=True)
parser.add_argument('--model_type', type=str, choices=['7b-Instruct', '13b-Instruct'])
parser.add_argument('--max_seq_len', type=int, default=2048)

args = parser.parse_args()
ckpt_dir = cache_dir + "/codellama/CodeLlama-{}".format(args.model_type)
tokenizer_path = ckpt_dir + "/tokenizer.model"
model: Llama = Llama.build(
ckpt_dir=ckpt_dir,
tokenizer_path=tokenizer_path,
max_seq_len=args.max_seq_len,
max_batch_size=1,
)

idx = args.testcase
test_llm_v2(icall_contexts[idx], function_declarators[idx], model)

if __name__ == '__main__':
main()
```

 运行的命令是 `torchrun --nproc-per-node 1 codellama_chat_test.py --testcase=1 --model_type=7b-Instruct --max_seq_len=1024`。

服务端用的脚本为

```python
headers = {
"Content-Type": "application/json"
}

prompt = [
{"role": "system",
"content": SystemPrompt1},
{"role": "user",
"content": UserPrompt1.format(icall_context[-1], "\n".join(icall_context),
"\n\n".join(func_name2declarator.values()))}
]

data = {
"model": "codellama",
"messages": prompt
}
# 定义server端的URL
server_url = "http://127.0.0.1:8888/v1/chat/completions" # 替换为实际的服务器地址和端口

# 发送POST请求，将JSON数据发送到server
response = requests.post(server_url, headers=headers, data=json.dumps(data))

# 检查服务器的响应状态码
if response.status_code == 200:
# 解析服务器的字符串响应
print("Response from server:", response_data)
```

### Error traceback

_No response_