Is the OpenAI-compatible API sending the `functions` argument to the model?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
I'm currently using the OS model functionary, which supports functions in a manner similar to how GPT operates through the OpenAI API. I've successfully deployed the model worker and proceeded to make chat completion requests using the functions feature:
curl {API}/chat/completions -H "Content-Type: application/json" -d '{
"model": "functionary-7b",
"messages": [{"role": "user", "content": "salute Anthony"}],
"functions": [
{"name": "salute",
"description": "This function can be used to salute someone.",
"parameters": {
"properties": {
"who": {
"description": "Name of whom to salute. o7",
"title": "Who",
"type": "string"
}
},
"required": [
"who"
],
"title": "salute",
"type": "object"
}
}]
}'
The expected outcome was to receive a message that included the function_call argument, indicating the use of the salute function with the argument Anthony. However, the actual result was as follows:
{"id":"chatcmpl-fUcuEKMeCkavtjsPCzeKft","object":"chat.completion","created":1694683275,"model":"functionary-7b","choices":[{"index":0,"message":{"role":"assistant","content":"\nSalute!\n"},"finish_reason":"stop"}],"usage":{"prompt_tokens":579,"total_tokens":585,"completion_tokens":6}}
It appears that the functions argument is being ignored altogether. My question is whether fastchat/serve/openai_api_server.py is indeed sending the functions argument to the model.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with fastchat/serve/openai_api_server.py and trace how the OpenAI-compatible chat completion request handles the functions field. Reproduce the provided curl request against the functionary-7b model, then verify whether functions reaches the model and whether the response includes the expected function_call data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100