abetlen / abetlen/llama-cpp-python

TypeError: 'add_special_tokens' is an invalid keyword argument for encode()

Đang mở
#1,250 0 bình luận 1 reaction 0 người được giao Xem trên GitHub
bug
Ngôn ngữ chính
Python
Star
10.6k
Fork
1.4k
Chỉ số merge pull request
Chỉ số pull request đang chờ

Mô tả

# Prerequisites
Trying to make use of chat completion for function calling and getting an invalid keyword argument.
I'm in Linux
```

`from llama_cpp import Llama
import code

from llama_cpp.llama_tokenizer import LlamaHFTokenizer
llm = Llama(
#model_path = "/mnt/opt5/experiment/a_dolphin-2.6-mistral-7b.Q6_K.gguf",
model_path = "/mnt/opt5/experiment/zoo/functionary-7b-v2.1.q8_0.gguf",
n_gpu_layers = -1, n_ctx = 2048,
chat_format="chatml-function-calling",
tokenizer=LlamaHFTokenizer("/mnt/opt5/experiment/hf_repos/functionary-7b-v2.1_tokenizer"),
verbose=True
)

messages = [
{"role": "user", "content": "what's the weather like in Hanoi?"}
]
tools = [ # For functionary-7b-v2 we use "tools"; for functionary-7b-v1.4 we use "functions" = [{"name": "get_current_weather", "description":..., "parameters": ....}]
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g., San Francisco, CA"
}
},
"required": ["location"]
}
}
}
]

result = llm.create_chat_completion(
messages = messages,
tools=tools,
tool_choice="auto",
)

print(result["choices"][0]["message"])
`

Running the code produces

Traceback (most recent call last):
File "/home/seg/code/2.py", line 37, in
result = llm.create_chat_completion(
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 1638, in create_chat_completion
return handler(
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama_chat_format.py", line 2451, in chatml_function_calling
completion_or_chunks = llama.create_completion(
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 1474, in create_completion
completion: Completion = next(completion_or_chunks) # type: ignore
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 913, in _create_completion
self.tokenize(prompt.encode("utf-8"), special=True)
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 469, in tokenize
return self.tokenizer_.tokenize(text, add_bos, special)
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama_tokenizer.py", line 75, in tokenize
return self.hf_tokenizer.encode(
TypeError: 'add_special_tokens' is an invalid keyword argument for encode()

Example environment info:
```
(llamapy) seg@seg-HP-Z820-Workstation:~/llama-cpp-python$ git log | head -3
commit 87a6e5797eb7b0cd63ad27c528fb950c80c84ad8
Author: Andrei Betlen
Date: Sun Mar 3 11:27:04 2024 -0500

(llamapy) seg@seg-HP-Z820-Workstation:~/llama-cpp-python$ python3 --version
Python 3.10.12

```

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.