feat(llm): investigate Tool search / defer_loading on the Responses path (GPT-5.4+)

📄 Study page: Tool search (defer_loading) on the OpenHands SDK

Summary

OpenAI's Tool search lets the model load tool definitions on demand instead of sending them all up front, cutting prompt tokens while preserving the prompt cache (loaded tools are appended at the end of the context window). Supported on gpt-5.4+. This issue tracks investigating and supporting it on the SDK's Responses path.

What the feature needs (from the OpenAI docs)

• Add a {"type": "tool_search"} tool to the tools array.
• Mark deferred tools with defer_loading: true (on functions, on the MCP server tool definition, or on functions inside a namespace). Namespaces/MCP servers are the recommended surface; keep each namespace < ~10 functions.
• Two modes:
  • Hosted: OpenAI searches the deferred tools declared in the request and returns the loaded subset.
  • Client-executed: model emits a tool_search_call; the app performs lookup and returns a tool_search_output (echoing call_id).
• Advanced: additional_tools input item (role developer) to inject tools at a specific point; namespace tool grouping.

Why it matters for OpenHands

An OpenHands agent typically loads a fat toolset (bash, str_replace_editor, browser, task/delegate, plus one function per MCP tool). Those schemas are re-sent every turn on the Responses path. Deferring rarely-used tools (especially large MCP surfaces) behind tool search could materially cut input tokens without losing capability.

Gaps in the current SDK

• No defer_loading flag anywhere; to_responses_tool() always emits the full parameter schema. → openhands-sdk/openhands/sdk/tool/tool.py:497
• No namespace grouping and no tool_search tool injection in _prepare_responses_params. → openhands-sdk/openhands/sdk/llm/llm.py:1258
• No parsing/handling of tool_search_call / tool_search_output items, and no additional_tools input support. → openhands-sdk/openhands/sdk/llm/message.py:541, openhands-sdk/openhands/sdk/llm/utils/responses_serialization.py

Proposed scope for a first cut

1. Add an opt-in defer_loading marker on tools + optional namespace grouping.
2. Inject the tool_search hosted tool when any deferred tool is present (start with hosted mode; it's the simplest).
3. Preserve/replay tool_search_call / tool_search_output items for store=false continuation.
4. Model-feature gate (gpt-5.4+).

Notes / open questions

• Depends on LiteLLM passing defer_loading, namespace, and tool_search through unchanged, and on surfacing the new item types.
• Interaction with PTC: deferred tools are not available to a program until loaded; the model must load them before starting a program (see companion PTC issue).

Investigation for GPT‑5.6 feature parity. Companion issues: programmatic tool calling, WebSocket mode, multi-agent.

Companion investigation issues (GPT‑5.6 Responses features): #4082 (Programmatic Tool Calling) · #4083 (Tool search) · #4084 (WebSocket mode) · #4085 (Multi-agent)

Read the linked Tool search study and inspect to_responses_tool() in openhands-sdk/openhands/sdk/tool/tool.py:497, _prepare_responses_params in openhands-sdk/openhands/sdk/llm/llm.py:1258, and the response item handling in message.py:541 and responses_serialization.py. Done means the Responses path supports the proposed hosted-mode deferred tools, feature gating, and preservation of tool_search_call/tool_search_output for store=false continuation.