microsoft / microsoft/vscode

Chat requests fail silently when tool definitions exceed the model's context budget (`budget=1`)

Open
#329,040 0 comments 0 reactions 1 assignee Claimed by @connor4312 View on GitHub
agent-host-mitigated bug
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

Does this issue occur when all extensions are disabled?: No

Note on the above: this is not an extension-caused bug, and bisect is not applicable. The trigger is the *aggregate size of registered tool definitions*, which comes primarily from MCP servers configured in `mcp.json` plus tool-contributing extensions. Disabling all extensions removes enough of that surface that the threshold is no longer crossed. The defect is in how Copilot Chat's prompt renderer handles the overflow, not in any single extension.

- VS Code Version:
Version: 1.131.0
Commit: e4c7e7b1d6d060162f4aa7f8225271b67ce1df75
Date: 2026-07-28T10:51:25Z
Electron: 42.7.0
ElectronBuildId: 14710161
Chromium: 148.0.7778.280
Node.js: 24.18.0
V8: 14.8.178.38-electron.0
OS: Darwin arm64 25.6.0
- OS Version: macOS Tahoe 26.6 (25G72)
- Copilot plan: Enterprise

Steps to Reproduce:

1. Configure MCP servers with a large aggregate tool surface. In my case, three servers (`@azure-devops/mcp`, `@softeria/ms-365-mcp-server --org-mode`, Notion over http) plus built-in and extension-contributed tools produced 475–509 tools totalling ~390–396k tokens of definitions.
2. Open Copilot Chat in Agent mode and select a model with a 200k context window.
3. Send any message.
4. Observe: spinner shows "thinking" for one to two seconds, then stops. No response, no error, nothing in the chat.

Reproduced reliably: three consecutive attempts within 36 seconds all produced `budget=1` and all failed silently.

---

## Summary

When registered tool definitions exceed a model's prompt budget, Copilot Chat computes a remaining budget of `1` token, **dispatches the request anyway**, and the request fails with no error surfaced anywhere in the UI. Nothing is logged at the default log level either, so the failure is completely invisible without raising the log level to Trace.

## Expected behavior

An actionable error, for example: "Tool definitions (233,783 tokens) exceed this model's context budget (200,000). Reduce the number of enabled tools or select a model with a larger context window."

## Actual behavior

Brief spinner, then silence.

## Diagnostic log line

At Trace level, one line explains the entire failure. Three consecutive failed requests from the same session:

```
2026-08-04 12:58:15.796 [debug] [Agent] rendering with budget=1 (baseBudget: 200000, toolTokens: 395969, totalTools: 509, toolSearchEnabled: true), summarizationEnabled=true
2026-08-04 12:58:21.065 [debug] [Agent] rendering with budget=1 (baseBudget: 200000, toolTokens: 391722, totalTools: 475, toolSearchEnabled: true), summarizationEnabled=true
2026-08-04 12:58:51.171 [debug] [Agent] rendering with budget=1 (baseBudget: 200000, toolTokens: 391722, totalTools: 475, toolSearchEnabled: true), summarizationEnabled=true
```

Tool definitions alone (391–396k tokens) roughly double the base budget of 200,000, leaving `budget=1` for the actual prompt. An earlier configuration failed the same way at a smaller overage:

```
[Agent] rendering with budget=1 (baseBudget: 200000, toolTokens: 233783, totalTools: 425, toolSearchEnabled: true), summarizationEnabled=true
```

In every case the request is then built and sent regardless:

```
[trace] Built prompt
[trace] Sending prompt to model
```

After that, nothing.

## Why this presented as a per-model entitlement problem

Only Claude Sonnet 4.6 worked; Opus 4.8 and Sonnet 5 failed silently. Sonnet 4.6 was configured with a larger context window, so the same ~234k of tool overhead still left usable budget. This looked exactly like a model policy or entitlement issue, and I spent significant time checking org-level model policies, seat assignment, and premium request quota before finding the cause. Quota was confirmed to be a non-factor:

```
[ChatQuota] processQuotaHeaders: {"quota":-1,"unlimited":true,"hasQuota":true,"percentRemaining":100,...}
```

## Confirmation of cause

After reducing the tool surface (domain filtering via `-d` on the ADO server, `--discovery` on the M365 server, removing a duplicate server registration, disabling one tool-contributing extension), the identical setup works on the same 200k model:

```
[Agent] rendering with budget=110761 (baseBudget: 200000, toolTokens: 76932, totalTools: 196, toolSearchEnabled: true)
```

## Related issues

I could not find an existing report of this failure mode. The closest concern the 128-tool cap, and the contrast (silence) is the point:

- microsoft/vscode#290356 "Hard tool limit (128) blocks agent mode when multiple MCP servers are enabled." A cap enforced at request time; the user must manually disable tools before proceeding. **Visibly blocked.**
- microsoft/vscode-copilot-release#13065 "Reason: You may not include more than 128 tools in your request." **Clean error message.**
- microsoft/vscode#277813 silent sub-agent failures, but these at least surface "Sorry, no response was returned."

So the *count*-based overflow path has user-facing error handling. The *token-budget* overflow path documented here has none. Whatever surfaces the 128-tool message seems like the natural place to also catch `budget` falling below a usable threshold.

## Secondary observation 1: `toolSearchEnabled: true` did not prevent the overflow

Tool search / deferred loading was enabled throughout. In the working configuration, 121 of 196 tools are deferred, so the mechanism does function. But it did not prevent ~390k of definitions from being charged against a 200k budget. Either deferred definitions are still counted in `toolTokens`, or the eager subset alone exceeded the window. Either way the budget accounting and the deferral mechanism appear to disagree.

Note also that between the first and second attempt above, the surface shrinks (509 → 475 tools, 395,969 → 391,722 tokens) while `budget` stays pinned at 1. Something adjusts the tool set between attempts, but by a margin nowhere near large enough to matter, and the request is dispatched anyway both times.

## Secondary observation 2: lockfile diffs are not excluded from chat context

Separately, every request in this workspace carried:

```
[GitDiffService] Truncating diff for .../package-lock.json (313195 chars -> 100000 chars)
[GitDiffService] Truncating diff for .../yarn.lock (445488 chars -> 100000 chars)
```

200,000 characters of lockfile diff per request, after truncation. Truncating to 100k concedes the content is too large while still sending most of it.

Copilot code review already excludes these files by name; the published exclusion list includes `package-lock.json`, `yarn.lock`, `Gemfile.lock`, `Cargo.lock`, `composer.lock`, and others. Applying the same list to `GitDiffService` working-tree diffs looks like a straightforward consistency fix. (Platform-level content exclusion is documented as unsupported in Edit and Agent modes, so it isn't a workaround here.)

## Suggested fixes

1. Fail fast with a clear message when `budget` falls below a usable threshold, rather than dispatching a request that cannot succeed.
2. Warn when `totalTools` or `toolTokens` approaches the budget, before the request is sent.
3. Reconcile `toolTokens` accounting with deferred tool loading when `toolSearchEnabled` is true.
4. Apply the code-review dependency-file exclusion list to working-tree diffs in chat context.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.