Azure / Azure/azure-rest-api-specs
[BUG] The file_search tool is often triggered incorrectly (Assistants API)
- Dominant language
- TypeSpec
- Stars
- 3.1k
- Forks
- 5.9k
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 444
Description
### API Spec link
https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/file-search?tabs=python
### API Spec version
2024-12-01
### Describe the bug
We have enabled the file_search tool for the Assistants API along with other custom tools, such as one for web search for example. However, when using the Azure Assistants API, the file_search tool gets triggered often, even when a web search would be the correct choice for a user query. There are days where everything works well in most of the cases and then there are day where it does almost every time choose the wrong tool.
If we remove the file_search tool, everything works as expected. Similarly, when using the OpenAI Assistants API with file_search enabled, everything also functions correctly (With exactly the same system prompt). But Azure behaves completely differently and unreliably, and it seems that the file_search tool is severely interfering with the correct function selection.
I can't understand why the Azure OpenAI Assistants API performs so much worse than OpenAI's. There are so many other unexplained issues, like the lack of image input support in Azure OpenAI, even though it works perfectly fine in OpenAI's API. Currently, the Azure Assistants API is completely unusable.
### Expected behavior
That the function call works just as reliably as in the equivalent OpenAI Assistants API.
### Actual behavior
The behavior is described above and should contain enough information. There is no stacktrace or something similar.
### Reproduction Steps
Activate file_search and add a custom function for web_search for example, instructing the model to use this tool for current events or questions about people. Then, ask a question about a recent event or a person.
It doesn't happen every time, but in many cases, the file_search tool is triggered instead of the intended web search.
### Environment
_No response_
Contributor guide
Research direction
Start by reviewing the linked Azure file-search API specification and following the reproduction steps with file_search and a custom web_search function. Compare the Azure Assistants API behavior with the equivalent OpenAI Assistants API behavior; done requires identifying an actionable specification or service-level cause for the incorrect tool selection.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, openapi
- Domain
- api, cloud
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100