microsoft / microsoft/vscode

Inline toolset reference is replaced by its first child tool in the model-facing chat request

Open
#330,764 0 comments 0 reactions 1 assignee Claimed by @aeschli View on GitHub
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

- Copilot Chat Extension Version: 0.61.0
- VS Code Version: 1.133.0
- OS Version: Windows Enterprise, build 26200
- Feature (e.g. agent/edit/ask mode): Agent mode
- Selected model (e.g. GPT 4.1, Claude 3.7 Sonnet): GPT 5.6 SOL

When a toolset is referenced inline, the model-facing chat request identifies the reference as the toolset's first child tool **rather than as the toolset itself**. The primary reproduction uses VS Code's built-in `#read` toolset in this repository and requires no extensions, MCP servers, or other dependencies.

## Primary Reproduction: Built-in `#read`

1. Open the VS Code repository, which contains a root `package.json`.
2. Start a new Agent mode chat.
3. Submit this prompt:

```text
Use #read to inspect package.json and summarize the scripts related to testing.
```

4. Inspect the model-facing chat request or request debug information.
5. Observe that the prompt is displayed correctly in the chat UI, but the ranged tool reference is identified as:

```text
copilot_getNotebookSummary
```

`getNotebookSummary` is the first child displayed under the `read` toolset. It was not explicitly selected in the prompt.

## Secondary Case: MCP Server Toolset

This case is not required to reproduce the bug. It illustrates that MCP server toolsets reach the same parser branch as the built-in `read` toolset.

1. Configure a hypothetical MCP server named `sample-mcp` that exposes `search` as its first tool and `read` as its second tool.
2. Start a new Agent mode chat and submit a prompt that references the whole server:

```text
Use #sample-mcp to find where ChatRequestToolSetPart is constructed.
```

3. The MCP contribution registers `sample-mcp` as a `ToolSet`, just as VS Code registers the built-in `read` group as a `ToolSet`.
4. The parser gives the `#sample-mcp` source range to both expanded child entries.
5. After the extension host flattens the toolset, the generated tool ID for `search`, as the first child, can be identified as the explicit ranged reference instead of `#sample-mcp`.

Both cases execute the same code in `ChatRequestParser.tryToParseVariable`; the toolset's source is not considered by that branch.

## Expected Behavior

The source range should identify the explicitly referenced toolset (`#read` or `#sample-mcp`). Its child tools should be made available to the model, but they should not claim the inline token's range.

## Actual Behavior

The toolset and every expanded child tool receive the same source range. When these entries are flattened for the extension API, the first child tool can be treated as the explicit inline reference.

## Root Cause

In `ChatRequestParser.tryToParseVariable`, the inline toolset is expanded by constructing every child with the parent token's ranges:

```ts
const value = Array.from(toolset.getTools()).map(t => new ChatRequestToolPart(varRange, varEditorRange, t.toolReferenceName ?? t.displayName, t.id, t.displayName, t.icon).toVariableEntry());
```

`extHostTypeConverters.ts` later flattens those child entries and preserves their ranges. This produces several references for the same prompt location, allowing the first child to masquerade as the inline reference.

The child entries should remain available but should not inherit the parent toolset's source range:

```ts
const value = Array.from(toolset.getTools()).map(t => toToolVariableEntry(t));
```

This is also consistent with how toolsets added as attachments create their child entries. Direct references to an individual `#tool` should continue to retain their own range.

## Complete Proposed Fix

In `src/vs/workbench/contrib/chat/common/requestParser/chatRequestParser.ts`, import the existing helper:

```ts
import { toToolVariableEntry } from '../attachments/chatVariableEntries.js';
```

Then replace the child construction in `ChatRequestParser.tryToParseVariable`:

```diff
const toolset = toolSetsByName.get(name);
if (toolset) {
- const value = Array.from(toolset.getTools()).map(t => new ChatRequestToolPart(varRange, varEditorRange, t.toolReferenceName ?? t.displayName, t.id, t.displayName, t.icon).toVariableEntry());
+ const value = Array.from(toolset.getTools()).map(t => toToolVariableEntry(t));
return new ChatRequestToolSetPart(varRange, varEditorRange, toolset.id, toolset.referenceName, toolset.icon, value);
}
```

The `ChatRequestToolSetPart` retains `varRange` and `varEditorRange`, while its expanded child entries are created without ranges.

## Regression Test

In `src/vs/workbench/contrib/chat/test/common/requestParser/chatRequestParser.test.ts`, apply these import changes:

```diff
+import { Codicon } from '../../../../../../base/common/codicons.js';
import { Event } from '../../../../../../base/common/event.js';

-import { ChatRequestAgentSubcommandPart, ChatRequestDynamicVariablePart, getPromptText } from '../../../common/requestParser/chatParserTypes.js';
+import { ChatRequestAgentSubcommandPart, ChatRequestDynamicVariablePart, ChatRequestToolSetPart, getPromptText } from '../../../common/requestParser/chatParserTypes.js';
```

Add this test to the `ChatRequestParser` suite:

```ts
test('inline toolset keeps the prompt range on the toolset only', () => {
const source = ToolDataSource.Internal;
const toolSet = new ToolSet(
'read',
'read',
Codicon.book,
source,
undefined,
undefined,
undefined,
false,
false,
instantiationService.get(IContextKeyService),
);
testDisposables.add(toolSet.addTool({ id: 'copilot_getNotebookSummary', toolReferenceName: 'getNotebookSummary', canBeReferencedInPrompt: true, displayName: 'getNotebookSummary', modelDescription: '', source }));
testDisposables.add(toolSet.addTool({ id: 'copilot_readFile', toolReferenceName: 'readFile', canBeReferencedInPrompt: true, displayName: 'readFile', modelDescription: '', source }));
variableService.setSelectedToolAndToolSets(testSessionUri, ToolAndToolSetEnablementMap.fromEntries([[toolSet, true]]));

parser = instantiationService.createInstance(ChatRequestParser);
const result = parser.parseChatRequest(testSessionUri, 'use #read for this');
const part = result.parts.find((candidate): candidate is ChatRequestToolSetPart => candidate instanceof ChatRequestToolSetPart);

assert.deepStrictEqual({
promptText: part?.promptText,
range: part?.range && { start: part.range.start, endExclusive: part.range.endExclusive },
tools: part?.tools.map(tool => ({ id: tool.id, range: tool.range })),
}, {
promptText: '#read',
range: { start: 4, endExclusive: 9 },
tools: [
{ id: 'copilot_getNotebookSummary', range: undefined },
{ id: 'copilot_readFile', range: undefined },
],
});
});
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.