huggingface / huggingface/tokenizers
apply_chat_template api usage consult
- Dominant language
- Rust
- Stars
- 11k
- Forks
- 1.2k
- Avg merge
- 3d 8h
- Merged PRs (30d)
- 26
Description
` environment: transformers=4.42.4, tokenizers=0.19.1`
I want to use apply_chat_template interface to implement tools calling。 Using the tools parameter to pass in function information, but the return result of the `apply_chat_template `interface does not add input tools info to the generated prompt.
how to fix this problem? Here is the code I used。
```python3
from transformers import AutoTokenizer
get_current_weather = {
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather in a given location.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["location"],
},
}
}
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "help me query currnet weather in San Francisco."},
]
tokenizer = AutoTokenizer.from_pretrained("xxxx/Qwen2-1.5B-Instruct")
prompt = tokenizer.apply_chat_template(
messages,
tools=[get_current_weather,],
tokenize=False,
add_generation_prompt=True,
)
print("prompt: ", prompt)
```
```
prompt: <|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
help me query currnet weather in San Francisco.<|im_end|>
<|im_start|>assistant
```
Contributor guide
Assessment
This issue has not been assessed yet.