microsoft / microsoft/vscode

Make agent-host tool search model allowlist configurable (or auto-trim tools for BYOK models like DeepSeek)

Open
#329,337 0 comments 0 reactions 1 assignee Claimed by @bhavyaus View on GitHub
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

## Summary

The agent host's **tool search (deferred tool loading)** feature is gated behind a hardcoded model allowlist (`agentHostModelSupportsToolSearch`). BYOK / third-party models (e.g. DeepSeek V4 registered via the `vizards.deepseek-v4-for-copilot` extension) are not in that allowlist, so agent-host sessions send **every enabled tool definition** to the model in a single request. Models with a per-request function-count limit (DeepSeek caps at 128) then fail with `502`.

## Problem

When using a BYOK model (e.g. DeepSeek V4 Flash) in a **Copilot agent host** session with many tools enabled (>128, commonly caused by MCP servers), the request fails:

```
Failed to get response from the AI model; retried 5 times
Last error: 502 DeepSeek 单次 tools 请求最多支持 128 个 functions,当前请求包含 203 个。
```

### Root cause

In `src/vs/platform/agentHost/node/copilot/toolSearchDeferral.ts`, `agentHostModelSupportsToolSearch(modelId)` is a hardcoded allowlist that only matches:

- GPT-5.4 / 5.5 / 5.6 variants
- Claude families 4.5 and newer

The launcher gates tool search on it (`copilotSessionLauncher.ts`):

```ts
const toolSearchActive = this._configurationService.getRootValue(copilotCliConfigSchema, CopilotCliConfigKey.ToolSearchEnabled) === true
&& agentHostModelSupportsToolSearch(effectiveModel?.id)
&& clientToolNames.has(CLIENT_TOOL_SEARCH_REFERENCE_NAME);
```

When the model is not in the allowlist, MCP and non-core tools are **not deferred**, and the SDK sends all tools up front — exceeding third-party API limits.

## Suggested changes (any one)

1. **Make the allowlist configurable** — e.g. extend `chat.agentHost.modelCapabilityOverrides` to accept a `toolSearch: boolean` capability field, or add a new setting like `chat.agentHost.toolSearch.models: string[]` (allowlist of model id prefixes) / `chat.agentHost.toolSearch.defaultForByok: boolean`.
2. **Default-enable tool search for BYOK models** — BYOK providers register through the LM API and typically support function calling (OpenAI-compatible), which is all the runtime `tool_search_tool` requires. Since BYOK is already an explicit opt-in (`chat.agentHost.byokModels.enabled`), defaulting tool search on for them is low risk.
3. **Advertise a per-model tool-count limit** through the BYOK bridge (`IByokLmModelInfo`) and have the SDK trim/group tools to fit it.

## Current workaround (verified NOT viable for BYOK)

Family aliasing is **insufficient** for BYOK models. Verified with agent-host logs:

```
Model capability override: routing prompt for 'deepseek/deepseek-v4-flash' as family 'claude-sonnet-4-6'
... still 502 DeepSeek 单次 tools 请求最多支持 128 个 functions,当前请求包含 204 个
```

The alias only affects VS Code's local capability layer. The **wire model id is unaffected** (`deepseek-v4-flash`), and the (closed-source) Copilot SDK runtime decides whether to honor `defer` based on that wire id — DeepSeek is not in its tool-search support set, so it ignores `defer: 'auto'` and forwards every tool. Both aliases below were tried; the 502 persisted.

```json
{
"chat.modelCapabilityOverrides": {
"deepseek-v4-flash": { "family": "claude-sonnet-4-6" }
},
"chat.agentHost.modelCapabilityOverrides": {
"deepseek/deepseek-v4-flash": { "family": "claude-sonnet-4-6" }
}
}
```

Side effects (avoidable): the aliased family routes the system prompt and every Claude-family heuristic (multi-replace tools, context editing, PDF support, …) to a model that is not Claude.

**The real fix must live in VS Code / the agent host** (see Suggested changes above). The most promising option is #3 — have the BYOK bridge advertise a per-model tool-count limit (`IByokLmModelInfo`) and let the SDK trim/group tools to fit it — because it requires no model-specific allowlists and works for any BYOK provider.

## References

- `extensions/copilot/src/extension/intents/node/agentIntent.ts` — `allowTools[CUSTOM_TOOL_SEARCH_NAME] = !!model.supportsToolSearch`
- `extensions/copilot/src/platform/endpoint/node/chatEndpoint.ts` — `this.family = getModelCapabilityOverride(...)?.family ?? ...`
- `extensions/copilot/src/platform/endpoint/common/chatModelCapabilities.ts` — `modelSupportsToolSearch`
- `src/vs/platform/agentHost/node/copilot/toolSearchDeferral.ts` — `agentHostModelSupportsToolSearch` (hardcoded allowlist)
- `src/vs/platform/agentHost/node/copilot/copilotSessionLauncher.ts` — tool-search gating in `_buildSessionConfig`

## Environment

- VS Code: latest stable (Insiders also affected)
- Extension: `Vizards.deepseek-v4-for-copilot` (BYOK, vendor `deepseek`, models `deepseek-v4-flash` / `deepseek-v4-pro`)
- Settings: `chat.agentHost.byokModels.enabled: true`
- Related source:
- `src/vs/platform/agentHost/node/copilot/toolSearchDeferral.ts`
- `src/vs/platform/agentHost/node/copilot/copilotSessionLauncher.ts` (`_buildSessionConfig`)
- `src/vs/platform/agentHost/common/copilotCliConfig.ts` (`chat.agentHost.modelCapabilityOverrides`)

## Repro

1. Install a BYOK extension that registers DeepSeek (or any model whose API limits functions per request).
2. Enable enough tools (>128, e.g. via MCP servers) so the agent host forwards more than the model's function limit.
3. Start an agent host session with the BYOK model and send a request.
4. Observe `502 ... tools ... limit` from the model provider after 5 retries.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.