abetlen / abetlen/llama-cpp-python
TypeError: 'add_special_tokens' is an invalid keyword argument for encode()
- 主要语言
- Python
- 星标
- 10.6k
- 派生
- 1.4k
- PR 合并指标
- PR 指标待抓取
描述
# Prerequisites
Trying to make use of chat completion for function calling and getting an invalid keyword argument.
I'm in Linux
```
`from llama_cpp import Llama
import code
from llama_cpp.llama_tokenizer import LlamaHFTokenizer
llm = Llama(
#model_path = "/mnt/opt5/experiment/a_dolphin-2.6-mistral-7b.Q6_K.gguf",
model_path = "/mnt/opt5/experiment/zoo/functionary-7b-v2.1.q8_0.gguf",
n_gpu_layers = -1, n_ctx = 2048,
chat_format="chatml-function-calling",
tokenizer=LlamaHFTokenizer("/mnt/opt5/experiment/hf_repos/functionary-7b-v2.1_tokenizer"),
verbose=True
)
messages = [
{"role": "user", "content": "what's the weather like in Hanoi?"}
]
tools = [ # For functionary-7b-v2 we use "tools"; for functionary-7b-v1.4 we use "functions" = [{"name": "get_current_weather", "description":..., "parameters": ....}]
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g., San Francisco, CA"
}
},
"required": ["location"]
}
}
}
]
result = llm.create_chat_completion(
messages = messages,
tools=tools,
tool_choice="auto",
)
print(result["choices"][0]["message"])
`
Running the code produces
Traceback (most recent call last):
File "/home/seg/code/2.py", line 37, in
result = llm.create_chat_completion(
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 1638, in create_chat_completion
return handler(
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama_chat_format.py", line 2451, in chatml_function_calling
completion_or_chunks = llama.create_completion(
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 1474, in create_completion
completion: Completion = next(completion_or_chunks) # type: ignore
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 913, in _create_completion
self.tokenize(prompt.encode("utf-8"), special=True)
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 469, in tokenize
return self.tokenizer_.tokenize(text, add_bos, special)
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama_tokenizer.py", line 75, in tokenize
return self.hf_tokenizer.encode(
TypeError: 'add_special_tokens' is an invalid keyword argument for encode()
Example environment info:
```
(llamapy) seg@seg-HP-Z820-Workstation:~/llama-cpp-python$ git log | head -3
commit 87a6e5797eb7b0cd63ad27c528fb950c80c84ad8
Author: Andrei Betlen
Date: Sun Mar 3 11:27:04 2024 -0500
(llamapy) seg@seg-HP-Z820-Workstation:~/llama-cpp-python$ python3 --version
Python 3.10.12
```
贡献指南
评估
这个 Issue 还没有评估数据。