Tool Search never defers MCP tools in the desktop app — gated behind an unassigned server experiment
- Dominant language
- No language data
- Stars
- 2.1k
- Forks
- 153
- PR merge metrics
- No merged PRs in 30d
Description
Investigated on macOS (App engine `1.0.80`, model `claude-opus-4.8`). MCP tool definitions load eagerly in every session regardless of model or config.
Measured (via `~/.copilot/data.db` → `sessions.context_tool_definitions_tokens`, on a copy): a fresh session whose only message was `hi` = 151,111 tool-def tokens. Same value across `claude-opus-4.8`, and up to 406k–440k on `gpt-5.6-sol` / `claude-opus-5` — so this is not model gating; all are "supported" models per the docs.
Root cause: deferral (`Session::apply_tool_search_defer_policy`, event `tool_search.activated`, present in the app binary) is gated behind the server-assigned `tool_search` experiment flight. My `app_state → exp-assignment-cache` (FlightingVersion 5389) only assigns `ccr_pr_nudge_adoption` + `app_aa_c1`; `tool_search` is absent. In the binary, `tool_search` sits in the same flight family as `mcp_apps` / `local_sandbox` / `cloud_sessions` — i.e. a server rollout, not a user toggle (which is why it also doesn't appear in Settings → Experimental).
Things that did NOT work:
* `deferTools: "auto"` per server in `mcp-config.json`
* Hard-adding `tool_search` to `app_state → enabled-experiments` in `data.db`
* `toolSearch: true` in `~/.copilot/settings.json`, tested both before and after a full app restart → tool-def tokens unchanged (151,111 → 151,115). The app constructs `toolSearch` itself from `expAssignments` and ignores this settings.json key, unlike the documented CLI.
Ask: please expose Tool Search as a user setting in the desktop app that bypasses the experiment gate, or enroll accounts in the `tool_search` flight — otherwise heavy MCP users pay for 150k–440k tool-definition tokens on every turn with no local mitigation.
---
Related: #1398 (dynamic MCP tool loading request), #2141 (large tool sets silently produce no output on Claude models in the app, while the CLI — where tool search is on by default — is unaffected). CLI docs: https://docs.github.com/en/copilot/concepts/agents/copilot-cli/tool-search
Contributor guide
Research direction
Start by examining the desktop binary's Session::apply_tool_search_defer_policy entry point, the tool_search.activated event, and the expAssignments flight data described in app_state → exp-assignment-cache. Compare this behavior with the CLI tool-search documentation and the documented settings paths. Done means the desktop app either offers a user-controlled bypass or enrolls accounts in tool_search, with the reported per-session tool-definition token count reduced.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos
- Domain
- ai-infra-agents, desktop
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100