microsoft / microsoft/vscode

BYOK Custom Endpoint: allow explicit Responses API tool search capability in chatLanguageModels.json

Open
#334,424 0 comments 0 reactions 1 assignee Claimed by @DonJayamanne View on GitHub
stale triage-needed
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

## Feature request

Allow BYOK models configured with the `customendpoint` provider and `apiType: "responses"` to explicitly opt into VS Code's existing client-executed Responses API tool search through `chatLanguageModels.json`.

## Problem

VS Code already has the Responses API tool-search implementation in place. In `extensions/copilot/src/platform/endpoint/node/responsesApi.ts`, tool search is enabled when `endpoint.supportsToolSearch` is true and the tool-search tool is available. The request is then shaped as the Responses API client-executed tool-search protocol (`{ type: "tool_search", execution: "client" }`), with deferred tools loaded through `tool_search_call` / `tool_search_output`.

However, `customendpoint` model configuration currently has no way to explicitly set that capability:

* `_CustomEndpointModelConfig` exposes fields such as `toolCalling`, `vision`, `thinking`, `adaptiveThinking`, and `supportsReasoningEffort`, but no tool-search capability.
* `BYOKModelCapabilities` / `resolveModelInfo()` do not populate `capabilities.supports.tool_search` from user configuration.
* `ChatEndpoint` already supports explicit metadata via `modelMetadata.capabilities.supports.tool_search`, but when it is absent it falls back to `modelSupportsToolSearch(this)`, which relies on known model IDs/families.

That fallback is useful for built-in/known models, but it is not a reliable authority for custom endpoints. A custom endpoint can proxy or rename a model, expose a future compatible model, or implement the Responses API tool-search protocol independently of a model ID that VS Code recognizes.

There is also `chat.modelCapabilityOverrides`, but that aliases the model family for capability routing in general. Using it just to enable tool search can unintentionally opt the custom model into other family-based behavior, so it is not a good replacement for a fine-grained capability flag.

## Proposed behavior

Add an optional per-model boolean in `chatLanguageModels.json`, for example:

```json
[
{
"vendor": "customendpoint",
"apiKey": "${input:chat.lm.secret.example}",
"models": [
{
"id": "my-responses-model",
"name": "My Responses Model",
"url": "https://example.com/v1/responses",
"apiType": "responses",
"toolCalling": true,
"toolSearch": true,
"vision": false,
"maxInputTokens": 200000,
"maxOutputTokens": 32000
}
]
}
]
```

Suggested semantics:

* `"toolSearch": true`: explicitly enable the existing tool-search path.
* `"toolSearch": false`: explicitly disable it, even if the model ID/family would otherwise match VS Code's built-in heuristic.
* omitted: preserve the current behavior and fall back to `modelSupportsToolSearch(...)`.

The three-state behavior fits the current `ChatEndpoint` logic because it already uses:

```ts
modelMetadata.capabilities.supports.tool_search ?? modelSupportsToolSearch(this)
```

## Possible implementation

The existing Responses API implementation should not need substantial changes. This could mainly be capability plumbing:

1. Add `toolSearch?: boolean` to `_CustomEndpointModelConfig`.
2. Add a corresponding optional field to `BYOKModelCapabilities`.
3. In `resolveModelInfo()`, preserve the optional value as `capabilities.supports.tool_search` rather than coercing it to `false`, so omission still falls back to the existing heuristic.
4. Pass the configured value through `CustomEndpointBYOKModelProvider.createOpenAIEndPoint()`.
5. Add the property to the `chatLanguageModels.json` schema in `extensions/copilot/package.json`.

For example:

```ts
tool_search: knownModelInfo?.toolSearch
```

rather than:

```ts
tool_search: !!knownModelInfo?.toolSearch
```

so `undefined` remains distinguishable from an explicit `false`.

## Why explicit configuration is preferable here

For a custom endpoint, VS Code cannot reliably determine support from the model name alone. The endpoint owner/user is in a better position to know whether the configured Responses API implementation accepts and correctly handles client-executed `tool_search`.

An explicit opt-in also avoids accidentally sending the special tool-search protocol to incompatible OpenAI-compatible endpoints, while still allowing proxies and custom/future models that do support it to use VS Code's existing tool deferral implementation.

I searched the existing VS Code issues for `tool_search`, BYOK, custom endpoint, and Responses API and did not find an issue covering this explicit custom-endpoint capability switch.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.