abetlen / abetlen/llama-cpp-python

TypeError: 'add_special_tokens' is an invalid keyword argument for encode()

未关闭
#1,250 0 条评论 1 个 reaction 已指派 0 人 在 GitHub 查看
bug
主要语言
Python
星标
10.6k
派生
1.4k
PR 合并指标
PR 指标待抓取

描述

# Prerequisites
Trying to make use of chat completion for function calling and getting an invalid keyword argument.
I'm in Linux
```

`from llama_cpp import Llama
import code

from llama_cpp.llama_tokenizer import LlamaHFTokenizer
llm = Llama(
#model_path = "/mnt/opt5/experiment/a_dolphin-2.6-mistral-7b.Q6_K.gguf",
model_path = "/mnt/opt5/experiment/zoo/functionary-7b-v2.1.q8_0.gguf",
n_gpu_layers = -1, n_ctx = 2048,
chat_format="chatml-function-calling",
tokenizer=LlamaHFTokenizer("/mnt/opt5/experiment/hf_repos/functionary-7b-v2.1_tokenizer"),
verbose=True
)

messages = [
{"role": "user", "content": "what's the weather like in Hanoi?"}
]
tools = [ # For functionary-7b-v2 we use "tools"; for functionary-7b-v1.4 we use "functions" = [{"name": "get_current_weather", "description":..., "parameters": ....}]
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g., San Francisco, CA"
}
},
"required": ["location"]
}
}
}
]

result = llm.create_chat_completion(
messages = messages,
tools=tools,
tool_choice="auto",
)

print(result["choices"][0]["message"])
`

Running the code produces

Traceback (most recent call last):
File "/home/seg/code/2.py", line 37, in
result = llm.create_chat_completion(
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 1638, in create_chat_completion
return handler(
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama_chat_format.py", line 2451, in chatml_function_calling
completion_or_chunks = llama.create_completion(
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 1474, in create_completion
completion: Completion = next(completion_or_chunks) # type: ignore
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 913, in _create_completion
self.tokenize(prompt.encode("utf-8"), special=True)
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 469, in tokenize
return self.tokenizer_.tokenize(text, add_bos, special)
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama_tokenizer.py", line 75, in tokenize
return self.hf_tokenizer.encode(
TypeError: 'add_special_tokens' is an invalid keyword argument for encode()

Example environment info:
```
(llamapy) seg@seg-HP-Z820-Workstation:~/llama-cpp-python$ git log | head -3
commit 87a6e5797eb7b0cd63ad27c528fb950c80c84ad8
Author: Andrei Betlen
Date: Sun Mar 3 11:27:04 2024 -0500

(llamapy) seg@seg-HP-Z820-Workstation:~/llama-cpp-python$ python3 --version
Python 3.10.12

```

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。