deepset-ai / deepset-ai/haystack

Agent generation_kwargs["tools"] replaces agent.tools instead of being treated as client-side specs

Open
#12,777 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
26.6k
Forks
3.2k
Avg merge
1d 3h
Merged PRs (30d)
194

Description

Describe the bug

When an Agent is given extra OpenAI tool specs in generation_kwargs["tools"] (typical for an OpenAI-compatible /chat/completions proxy that forwards the client tools catalog), those specs replace the Agent’s own tools on the chat-generator call. The model never sees agent.tools.

Sample chat completions payload with client-side tools cataloq (see tools->search_knowledge_files) :

{
  "stream": true,
  "model": "workspace/pipeline",
  "messages": [
    {
      "role": "user",
      "content": "What’s the capital of Germany?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "search_knowledge_files",
        "description": "Search files by filename across knowledge bases the user has access to.",
        "parameters": {
          "type": "object",
          "properties": {
            "query": {
              "type": "string",
              "description": "The search query to find matching files by filename"
            },
            "count": {
              "type": "integer",
              "default": 5,
              "description": "Maximum number of results to return"
            }
          },
          "required": ["query"]
        }
      }
    }
  ]
}

That happens because OpenAIChatGenerator._prepare_api_call builds the API payload as {**openai_tools, **generation_kwargs}. A tools key in generation_kwargs overwrites the list built from the tools argument.

After a generator-only merge, a second failure remains: if the model calls a client-only name, the Agent tries to invoke it and hits ToolNotFound (or an error tool message). Client tools should be advertised to the LLM and left for the caller to execute.

Expected behavior

The LLM is offered agent.tools and the extra OpenAI specs (client-side tools) .
The Agent invokes only tools it owns.
Calls whose names exist only in generation_kwargs["tools"] are left on the last assistant message so a client can execute them. A dedicated exit_reason (for example "client_tools") would make this easy to route.
Truly unknown names (not in either list) keep today’s ToolNotFound path.

To Reproduce

from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import Tool
def local_search(query: str) -> str:
    return f"kb:{query}"
agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-4o"),
    tools=[Tool(name="local_search", description="Search the knowledge base.", parameters={"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]}, function=local_search)],
)
agent.run(
    [ChatMessage.from_user("What is Alzheimer's?")],
    generation_kwargs={
        "tools": [{"type": "function", "function": {"name": "search_knowledge_files", "description": "Client catalog search."}}],
    },
)

The OpenAI request tools list contains only search_knowledge_files. local_search is missing.

Suggested fix

Pop tools from run-time generation_kwargs so they cannot overwrite the generator tools argument.
Convert those function specs into spec-only Haystack Tool stubs and pass current_tools + stubs into the chat generator.
Invoke only Agent-owned tools; stop when the model called a client-side name.
A complementary generator change (merge the two tools lists instead of last-write-wins) would still help OpenAIChatGenerator.run(tools=..., generation_kwargs={"tools": ...}) used without an Agent, including built-in OpenAI tools that cannot be expressed as Haystack Tool objects.

Draft implementation:
#12775

System

Haystack version: main
Component: Agent + OpenAIChatGenerator / OpenAIResponsesChatGenerator

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Agent.run and OpenAIChatGenerator._prepare_api_call, then compare the corresponding OpenAIResponsesChatGenerator behavior. Reproduce the request using the supplied generation_kwargs tools and inspect how tool calls are handled after generation. Done means agent tools and client-side specs are both advertised, only agent-owned tools are invoked, client-only calls remain available to the caller, and unknown names retain the existing ToolNotFound path.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.