googleapis / googleapis/python-genai

Feature Request: Support defer_loading and tool_search for dynamic tool discovery (parity with Anthropic & OpenAI)

Open
#2,185 1 comment 0 reactions 1 assignee Claimed by @Venkaiahbabuneelam View on GitHub
priority: p2 type: bug type: feature request
Dominant language
Python
Stars
4k
Forks
1k
Avg merge
2d 11h
Merged PRs (30d)
40

Description

## Summary

Both Anthropic (Claude Sonnet 4.0+) and OpenAI (GPT-5.4+) now support **deferred tool loading** and **tool search** at the API level. The Gemini API currently lacks this capability.

Same feature request as googleapis/js-genai#1416 — filing here for visibility across both SDKs since this is an API-level feature.

## Problem

When building AI agents with many tools (30+):

1. **Context bloat**: All tool schemas consume context tokens upfront. A typical multi-server MCP setup can consume 50k+ tokens in tool definitions alone.
2. **Tool selection accuracy**: Model accuracy degrades beyond 30-50 tools.
3. **Client-side workarounds break caching**: Pre-filtering tools before each API call changes the tools parameter, invalidating prefix cache.

## What competitors have

### Anthropic
- defer_loading: true on tools — schema sent but not rendered into model context
- tool_search_tool_regex/bm25 — model searches deferred tools, matched schemas expanded within same API call
- Zero extra round-trips, cache preserved
- Up to 10,000 tools

### OpenAI
- Same defer_loading: true mechanism
- Namespaces to group tools — model only sees group name + description
- Discovered schemas injected at end of context window to preserve cache
- Hosted and client-executed search modes

## Proposed for Gemini API

1. defer_loading: true on functionDeclarations
2. Built-in tool_search tool for on-demand discovery
3. Automatic schema expansion within generateContent call

## Related issues

- googleapis/js-genai#1416 (same request, JS SDK)
- googleapis/js-genai#435 (search + function calling cannot be combined)
- googleapis/js-genai#1157 (urlContext + functionDeclarations cannot be combined)
- google-gemini/gemini-cli#21823 (100 tool limit)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.