abetlen / abetlen/llama-cpp-python

TypeError: 'add_special_tokens' is an invalid keyword argument for encode()

Abierto
#1,250 0 comentarios 1 reacción 0 asignados Ver en GitHub
bug
Lenguaje dominante
Python
Estrellas
10.6k
Forks
1.4k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

# Prerequisites
Trying to make use of chat completion for function calling and getting an invalid keyword argument.
I'm in Linux
```

`from llama_cpp import Llama
import code

from llama_cpp.llama_tokenizer import LlamaHFTokenizer
llm = Llama(
#model_path = "/mnt/opt5/experiment/a_dolphin-2.6-mistral-7b.Q6_K.gguf",
model_path = "/mnt/opt5/experiment/zoo/functionary-7b-v2.1.q8_0.gguf",
n_gpu_layers = -1, n_ctx = 2048,
chat_format="chatml-function-calling",
tokenizer=LlamaHFTokenizer("/mnt/opt5/experiment/hf_repos/functionary-7b-v2.1_tokenizer"),
verbose=True
)

messages = [
{"role": "user", "content": "what's the weather like in Hanoi?"}
]
tools = [ # For functionary-7b-v2 we use "tools"; for functionary-7b-v1.4 we use "functions" = [{"name": "get_current_weather", "description":..., "parameters": ....}]
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g., San Francisco, CA"
}
},
"required": ["location"]
}
}
}
]

result = llm.create_chat_completion(
messages = messages,
tools=tools,
tool_choice="auto",
)

print(result["choices"][0]["message"])
`

Running the code produces

Traceback (most recent call last):
File "/home/seg/code/2.py", line 37, in
result = llm.create_chat_completion(
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 1638, in create_chat_completion
return handler(
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama_chat_format.py", line 2451, in chatml_function_calling
completion_or_chunks = llama.create_completion(
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 1474, in create_completion
completion: Completion = next(completion_or_chunks) # type: ignore
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 913, in _create_completion
self.tokenize(prompt.encode("utf-8"), special=True)
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama.py", line 469, in tokenize
return self.tokenizer_.tokenize(text, add_bos, special)
File "/home/seg/venv/llamapy/lib/python3.10/site-packages/llama_cpp/llama_tokenizer.py", line 75, in tokenize
return self.hf_tokenizer.encode(
TypeError: 'add_special_tokens' is an invalid keyword argument for encode()

Example environment info:
```
(llamapy) seg@seg-HP-Z820-Workstation:~/llama-cpp-python$ git log | head -3
commit 87a6e5797eb7b0cd63ad27c528fb950c80c84ad8
Author: Andrei Betlen
Date: Sun Mar 3 11:27:04 2024 -0500

(llamapy) seg@seg-HP-Z820-Workstation:~/llama-cpp-python$ python3 --version
Python 3.10.12

```

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.