abetlen / abetlen/llama-cpp-python

chatml-function-calling chat format fails to generate multi calls to the same tool

Đang mở
#1,756 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
10.6k
Fork
1.4k
Chỉ số merge pull request
Chỉ số pull request đang chờ

Mô tả

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [x] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [x] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [x] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.

# Expected Behavior

The response should return multiple calls to the tools function

# Current Behavior

A single call is generated.

# Environment and Context

Please provide detailed information about your computer setup. This is important in case the issue is not reproducible except for under certain specific conditions.

* Physical (or virtual) hardware you are using, e.g. for Linux: Win11Pro on laptop

# Failure Information (for bugs)

Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.

# Steps to Reproduce

Please provide detailed steps for reproducing the issue. We are not sitting in front of your screen, so the more detail the better.

Download model file from https://huggingface.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF/resolve/main/Mistral-7B-Instruct-v0.3.Q4_K_M.gguf

```

import os
from llama_cpp.llama import Llama

llm = Llama(
model_path="Mistral-7B-Instruct-v0.3.Q4_K_M.gguf", chat_format='chatml-function-calling'

)

SYSTEM_MESSAGE="""
You are a helpful assistant.
You can call functions with appropriate input when necessary.
You can call the same function several times.
"""

weather = {
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather in a given latitude and longitude",
"parameters": {
"type": "object",
"properties": {
"latitude": {
"type": "number",
"description": "The latitude of a place",
},
"longitude": {
"type": "number",
"description": "The longitude of a place",
},
},
"required": ["latitude", "longitude"],
},
},
}

question = "What's the weather like in the following cities: Sydney and Paris ?"
messages = [
{"role": "system", "content": SYSTEM_MESSAGE},
{"role": "user", "content": question}
]
response = llm.create_chat_completion_openai_v1(messages, tools=[weather],tool_choice='auto')
print(response)
```

# Failure Logs

N/A

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.