deepset-ai / deepset-ai/haystack
Agent generation_kwargs["tools"] replaces agent.tools instead of being treated as client-side specs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 26.6k
- Forks
- 3.2k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 194
Description
Describe the bug
When an Agent is given extra OpenAI tool specs in generation_kwargs["tools"] (typical for an OpenAI-compatible /chat/completions proxy that forwards the client tools catalog), those specs replace the Agent’s own tools on the chat-generator call. The model never sees agent.tools.
Sample chat completions payload with client-side tools cataloq (see tools->search_knowledge_files) :
{
"stream": true,
"model": "workspace/pipeline",
"messages": [
{
"role": "user",
"content": "What’s the capital of Germany?"
}
],
"tools": [
{
"type": "function",
"function": {
"name": "search_knowledge_files",
"description": "Search files by filename across knowledge bases the user has access to.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "The search query to find matching files by filename"
},
"count": {
"type": "integer",
"default": 5,
"description": "Maximum number of results to return"
}
},
"required": ["query"]
}
}
}
]
}
That happens because OpenAIChatGenerator._prepare_api_call builds the API payload as {**openai_tools, **generation_kwargs}. A tools key in generation_kwargs overwrites the list built from the tools argument.
After a generator-only merge, a second failure remains: if the model calls a client-only name, the Agent tries to invoke it and hits ToolNotFound (or an error tool message). Client tools should be advertised to the LLM and left for the caller to execute.
Expected behavior
The LLM is offered agent.tools and the extra OpenAI specs (client-side tools) .
The Agent invokes only tools it owns.
Calls whose names exist only in generation_kwargs["tools"] are left on the last assistant message so a client can execute them. A dedicated exit_reason (for example "client_tools") would make this easy to route.
Truly unknown names (not in either list) keep today’s ToolNotFound path.
To Reproduce
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import Tool
def local_search(query: str) -> str:
return f"kb:{query}"
agent = Agent(
chat_generator=OpenAIChatGenerator(model="gpt-4o"),
tools=[Tool(name="local_search", description="Search the knowledge base.", parameters={"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]}, function=local_search)],
)
agent.run(
[ChatMessage.from_user("What is Alzheimer's?")],
generation_kwargs={
"tools": [{"type": "function", "function": {"name": "search_knowledge_files", "description": "Client catalog search."}}],
},
)
The OpenAI request tools list contains only search_knowledge_files. local_search is missing.
Suggested fix
Pop tools from run-time generation_kwargs so they cannot overwrite the generator tools argument.
Convert those function specs into spec-only Haystack Tool stubs and pass current_tools + stubs into the chat generator.
Invoke only Agent-owned tools; stop when the model called a client-side name.
A complementary generator change (merge the two tools lists instead of last-write-wins) would still help OpenAIChatGenerator.run(tools=..., generation_kwargs={"tools": ...}) used without an Agent, including built-in OpenAI tools that cannot be expressed as Haystack Tool objects.
Draft implementation:
#12775
System
Haystack version: main
Component: Agent + OpenAIChatGenerator / OpenAIResponsesChatGenerator
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with Agent.run and OpenAIChatGenerator._prepare_api_call, then compare the corresponding OpenAIResponsesChatGenerator behavior. Reproduce the request using the supplied generation_kwargs tools and inspect how tool calls are handled after generation. Done means agent tools and client-side specs are both advertised, only agent-owned tools are invoked, client-only calls remain available to the caller, and unknown names retain the existing ToolNotFound path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend-api-design
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100