github / github/app

Tool Search never defers MCP tools in the desktop app — gated behind an unassigned server experiment

Aperta
#3,172 0 commenti 1 reazione 0 assegnatari Vedi su GitHub
Lingua principale
Nessun dato sulla lingua
Stelle
2.1k
Fork
153
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

Investigated on macOS (App engine `1.0.80`, model `claude-opus-4.8`). MCP tool definitions load eagerly in every session regardless of model or config.

Measured (via `~/.copilot/data.db` → `sessions.context_tool_definitions_tokens`, on a copy): a fresh session whose only message was `hi` = 151,111 tool-def tokens. Same value across `claude-opus-4.8`, and up to 406k–440k on `gpt-5.6-sol` / `claude-opus-5` — so this is not model gating; all are "supported" models per the docs.

Root cause: deferral (`Session::apply_tool_search_defer_policy`, event `tool_search.activated`, present in the app binary) is gated behind the server-assigned `tool_search` experiment flight. My `app_state → exp-assignment-cache` (FlightingVersion 5389) only assigns `ccr_pr_nudge_adoption` + `app_aa_c1`; `tool_search` is absent. In the binary, `tool_search` sits in the same flight family as `mcp_apps` / `local_sandbox` / `cloud_sessions` — i.e. a server rollout, not a user toggle (which is why it also doesn't appear in Settings → Experimental).

Things that did NOT work:

* `deferTools: "auto"` per server in `mcp-config.json`
* Hard-adding `tool_search` to `app_state → enabled-experiments` in `data.db`
* `toolSearch: true` in `~/.copilot/settings.json`, tested both before and after a full app restart → tool-def tokens unchanged (151,111 → 151,115). The app constructs `toolSearch` itself from `expAssignments` and ignores this settings.json key, unlike the documented CLI.

Ask: please expose Tool Search as a user setting in the desktop app that bypasses the experiment gate, or enroll accounts in the `tool_search` flight — otherwise heavy MCP users pay for 150k–440k tool-definition tokens on every turn with no local mitigation.

---

Related: #1398 (dynamic MCP tool loading request), #2141 (large tool sets silently produce no output on Claude models in the app, while the CLI — where tool search is on by default — is unaffected). CLI docs: https://docs.github.com/en/copilot/concepts/agents/copilot-cli/tool-search

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start by examining the desktop binary's Session::apply_tool_search_defer_policy entry point, the tool_search.activated event, and the expAssignments flight data described in app_state → exp-assignment-cache. Compare this behavior with the CLI tool-search documentation and the documented settings paths. Done means the desktop app either offers a user-controlled bypass or enrolls accounts in tool_search, with the reported per-session tool-definition token count reduced.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
macos
Ambito
ai-infra-agents, desktop
Tipo di issue
Funzionalità
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Attiva
Chiarezza
Abbastanza chiara
Idoneità per principianti
38/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.