microsoft / microsoft/agent-framework-go
Gemini: set includeServerSideToolInvocations when mixing function tools with native tools (needs backend-detection decision)
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 618
- Forks
- 56
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 122
Description
Porting parity item from the Python Gemini client, surfaced while comparing `provider/geminiprovider` against `python/packages/gemini`.
## Gap
On the Gemini **Developer API**, when a request combines `FunctionDeclarations` with a server-side/native tool (Google Search, code execution, file search, …), the Python client asks the server to echo its tool interactions:
```python
# _prepare_config
if tools and not self._vertexai and self._requires_server_side_tool_invocations(tools):
tool_config = tool_config or types.ToolConfig()
tool_config.include_server_side_tool_invocations = True
```
The Go provider never sets `genai.ToolConfig.IncludeServerSideToolInvocations` (the field exists — `types.go:2865` — but is unreferenced in the provider). Combined with the fact that Go also drops server-side `toolCall`/`toolResponse` response parts today (separate PR), a Go caller mixing a function tool with e.g. web search never observes the server-side tool invocations that Python surfaces.
## Why this needs a decision before a PR
Python gates the flag on `not self._vertexai`, because the field is **Developer-API-only** — the genai SDK itself errors if it is sent in Vertex mode (`models.go`). The Go provider cannot reproduce that gate cleanly:
- `genai.Client` stores its backend in the **unexported** `clientConfig` field and exposes **no accessor** (`func (c *Client) ...`) to read the backend or `ClientConfig`.
- So the provider has no supported way to know whether it is talking to the Developer API or Vertex AI at request-build time.
## Options
1. Add an `AgentConfig` field (e.g. `Backend genai.Backend`, or a narrower `UseVertexAI bool`) so callers can declare the backend the provider was constructed against, and gate the flag on it — mirroring Python’s `_vertexai`.
2. Only set the flag when the caller has explicitly opted in via a new config flag (no auto-detection), leaving backend correctness to the caller.
3. Upstream a backend accessor on `genai.Client` and gate on that (out of this repo’s control; slowest).
Option 1 seems closest to the .NET/Python behavior and to how `AgentConfig` already carries provider-construction context. Happy to implement whichever direction maintainers prefer — filing first per CONTRIBUTING since this adds public config surface.
## References
- Python: `python/packages/gemini/agent_framework_gemini/_chat_client.py` — `_prepare_config`, `_requires_server_side_tool_invocations`.
- SDK field: `google.golang.org/genai` `ToolConfig.IncludeServerSideToolInvocations` (Developer-API only).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Go provider construction and AgentConfig, then compare provider/geminiprovider with Python's _chat_client.py _prepare_config and _requires_server_side_tool_invocations. Read types.go:2865 and models.go to confirm the Developer-API restriction and backend-access limitation. Done means maintainers choose a backend-detection or opt-in approach before implementation proceeds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- ai, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100