microsoft / microsoft/vscode

Copilot subagents receive mandatory tool_search instructions although Responses API disables tool search for subagents

Open
#336,376 0 comments 0 reactions 1 assignee Claimed by @roblourens View on GitHub
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

## Summary

Copilot's Responses API request construction and tool-search prompt generation use inconsistent conditions for subagents.

The request builder intentionally disables tool deferral for subagents, provides ordinary tool schemas directly, and omits `tool_search`. However, prompt generation can still label those tools as deferred and instruct the subagent that it **must call `tool_search` before using them**.

This creates an impossible prerequisite: a subagent can stop because the required discovery tool is unavailable, even though the ordinary tool it needs has already been supplied.

**This is not a request to enable tool search for subagents.** Keeping subagent tools eagerly loaded is valid. The bug is retaining mandatory search instructions when search is disabled for that request.

## Environment and verification scope

- OS: Windows.
- VS Code: `1.137.0`.
- GitHub Copilot extension (`GitHub.copilot-chat`): `0.65.0`.
- The relevant conditions were present in GitHub `main` when inspected on 2026-09-16. The main ref resolved to `0b49fec181fe9415a9aa5b0315c32863b84a83d3` during report preparation.
- The installed Copilot bundle also contains the subagent exclusion, search-tool removal, and mandatory deferred-tool guidance described below.
- An observed subagent stopped with a missing-search-tool explanation. A diagnostic invocation reported ordinary tool definitions present, no callable search tool, and instructions requiring discovery before invocation.
- That diagnostic inventory is model-reported, not a captured network request. The exact model ID and transport for the failed invocation were not captured.
- Latest `main` has not been built and tested end to end. The analysis below establishes the source-level inconsistency; it does not claim an automated reproduction across all models or execution paths.

## Expected behavior

The prompt and serialized request should agree on the effective tool-loading policy:

- If tool deferral is disabled, provide the ordinary schemas without a deferred-tool list or mandatory search-before-call instructions.
- If tool deferral is enabled, provide the corresponding search capability and consistent discovery guidance.
- Preserve existing tool permissions, approval requirements, and invocation validation.

## Actual behavior

A subagent can receive both full ordinary tool definitions and instructions such as:

```text
You MUST use tool_search to load deferred tools BEFORE calling them.
Calling a deferred tool without loading it first will fail.
```

The global context can also contain:

```text
Available deferred tools (must be loaded with tool_search before use):
...
```

Yet the final Responses API request contains no search tool. Following these instructions can cause the subagent to stop without invoking an otherwise available tool.

## Source-level explanation

### 1. Request serialization excludes subagents from tool deferral

[createResponsesRequestBody in responsesApi.ts](https://github.com/microsoft/vscode/blob/main/extensions/copilot/src/platform/endpoint/node/responsesApi.ts#L69):

```ts
const toolSearchEnabled = !!endpoint.supportsToolSearch
&& !!options.requestOptions?.tools?.some(t => t.function.name === CUSTOM_TOOL_SEARCH_NAME);
const isAllowedConversationAgent = options.location === ChatLocation.Agent || options.location === ChatLocation.MessagesProxy;
const isSubagent = options.telemetryProperties?.subType?.startsWith('subagent') ?? false;
const shouldDeferTools = toolSearchEnabled && isAllowedConversationAgent && !isSubagent;
```

In the same function:

- The ordinary function named `CUSTOM_TOOL_SEARCH_NAME` is always skipped because `tool_search` is a reserved Responses API namespace.
- The protocol-level `{ type: 'tool_search', execution: 'client' }` tool is added only when `shouldDeferTools` is true.
- When deferral is disabled, ordinary tools are included with full definitions rather than withheld for discovery.

Consequently, subagents receive eagerly loaded ordinary tools and no search tool, even if search appeared in the pre-serialization tool set.

### 2. Shared prompt generation checks endpoint capability, not effective request policy

[ToolSearchToolPromptOptimized and DeferredToolListReminder](https://github.com/microsoft/vscode/blob/main/extensions/copilot/src/extension/prompts/node/agent/toolSearchInstructions.tsx#L47) do not use the same request-level condition.

The shared guidance checks:

```ts
if (!endpoint?.supportsToolSearch || !hasDeferredTool(this.props.availableTools, this.toolDeferralService)) {
return;
}
```

The deferred-tool list similarly checks endpoint support and classifies tools with `isNonDeferredTool`. These components do not check whether the current subagent request actually enables deferral.

### 3. Model-specific reminders repeat the prerequisite

[Gpt55ReminderInstructions](https://github.com/microsoft/vscode/blob/main/extensions/copilot/src/extension/prompts/node/agent/openai/gpt55Prompt.tsx) uses:

```ts
const toolSearchEnabled = !!this.props.endpoint.supportsToolSearch;
```

It then emits another mandatory search-before-invocation reminder. Fixing only the shared system guidance would leave this additional prompt surface inconsistent.

```text
Endpoint supports search
-> prompts label tools deferred and mandate discovery
Subagent request
-> serializer disables deferral and omits search
-> ordinary schemas are supplied directly
Result: prompt requirements contradict the available capabilities
```

## Reproduction and regression test scenario

The following is a proposed reduced test scenario, not a separately executed standalone reproducer:

1. Use a search-capable Responses API endpoint and an available harmless extension/MCP tool that the deferral service normally classifies as deferred.
2. Invoke a subagent with that tool enabled and ask it to call the tool.
3. Inspect the generated prompt and final serialized request together.
4. Verify that the subagent request includes the ordinary tool schema and no search tool, while the prompt still contains mandatory discovery guidance and/or the deferred-tool list.

The source-level mismatch is determined by the conditions above. Whether the model stops or attempts the supplied tool despite conflicting guidance can vary.

## Impact

- **Task execution can be blocked:** a subagent following the injected prerequisite may refuse to call an otherwise available tool.
- **Multi-agent workflows can stall:** dependent work cannot advance when a delegated task stops on a nonexistent discovery requirement.
- **Misleading diagnostics:** the symptom suggests missing tools, MCP registration failures, or invalid agent configuration even though ordinary tool schemas are already supplied.
- **Wasted time and requests:** retries and unnecessary tool/configuration troubleshooting consume time and model requests without advancing the task. No quantitative cost measurement is available.
- **Inconsistent behavior:** success can depend on how the model handles contradictory instructions rather than tool availability. This report does not assert that every model or subagent fails.
- **Scope:** potentially affects subagent workflows using extension/MCP tools under the identified Responses API conditions, not any particular application or business domain.
- **No observed data loss or security incident:** the observed failure was a stop before the desired tool invocation, not an application failure or an executed tool exception.

Adding tool-search entries to an agent configuration cannot override the explicit `!isSubagent` condition in this serialization path.

## Suggested fix direction

Use a shared, request-scoped effective search/deferral policy for both prompt generation and serialization. When deferral is disabled for a subagent, suppress the shared discovery guidance, misleading deferred-tool list, and model-specific reminders requiring discovery.

Checking only the pre-serialization presence of the search tool is insufficient if the serializer subsequently removes it.

Suggested regression coverage:

1. Subagent + search-capable endpoint + extension/MCP tool: full schema present, search absent, no mandatory discovery instructions or misleading deferred list.
2. Top-level agent + search enabled: preserve client-executed search and correct schema deferral.
3. Search disabled or unsupported: no search-before-call prerequisite.
4. Validate shared system guidance, global context, and model-specific reminders together.
5. End-to-end subagent invocation of a harmless tool to verify that execution advances.

## Related reports

- #317562 requested enabling search for subagents and was closed as not planned. This report instead concerns consistent prompts while search remains disabled.
- #313899, #315526, and #312414 describe older discovery/schema-availability issues. The specific problem here is mandatory discovery guidance remaining in subagent prompts when serialization disables search.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.