Azure / Azure/azure-rest-api-specs

[BUG] The file_search tool is often triggered incorrectly (Assistants API)

Open
#32,487 0 comments 1 reaction 0 assignees View on GitHub
bug customer-reported OpenAI question Service Attention
Dominant language
TypeSpec
Stars
3.1k
Forks
5.9k
Avg merge
2d 22h
Merged PRs (30d)
444

Description

### API Spec link

https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/file-search?tabs=python

### API Spec version

2024-12-01

### Describe the bug

We have enabled the file_search tool for the Assistants API along with other custom tools, such as one for web search for example. However, when using the Azure Assistants API, the file_search tool gets triggered often, even when a web search would be the correct choice for a user query. There are days where everything works well in most of the cases and then there are day where it does almost every time choose the wrong tool.

If we remove the file_search tool, everything works as expected. Similarly, when using the OpenAI Assistants API with file_search enabled, everything also functions correctly (With exactly the same system prompt). But Azure behaves completely differently and unreliably, and it seems that the file_search tool is severely interfering with the correct function selection.

I can't understand why the Azure OpenAI Assistants API performs so much worse than OpenAI's. There are so many other unexplained issues, like the lack of image input support in Azure OpenAI, even though it works perfectly fine in OpenAI's API. Currently, the Azure Assistants API is completely unusable.

### Expected behavior

That the function call works just as reliably as in the equivalent OpenAI Assistants API.

### Actual behavior

The behavior is described above and should contain enough information. There is no stacktrace or something similar.

### Reproduction Steps

Activate file_search and add a custom function for web_search for example, instructing the model to use this tool for current events or questions about people. Then, ask a question about a recent event or a person.

It doesn't happen every time, but in many cases, the file_search tool is triggered instead of the intended web search.

### Environment

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the linked Azure file-search API specification and following the reproduction steps with file_search and a custom web_search function. Compare the Azure Assistants API behavior with the equivalent OpenAI Assistants API behavior; done requires identifying an actionable specification or service-level cause for the incorrect tool selection.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, openapi
Domain
api, cloud
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.