microsoft / microsoft/vscode

When using custom endpoints, virtual tools cause the model to fail to accurately discover the tools

Open
#335,227 0 comments 0 reactions 1 assignee Claimed by @justschen View on GitHub
triage-needed
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

- Copilot Chat Extension Version: 1.0.81-0
- VS Code Version: 1.136.2
- OS Version: Windows 11 24H2
- Feature (e.g. agent/edit/ask mode): Tool call
- Selected model (e.g. GPT 4.1, Claude 3.7 Sonnet): Custom endpoint

Steps to Reproduce:

1. Use API Keys to access custom endpoint models(or OpenRouter), such as GPT 5.6 Sol
2. Connect to an MCP that includes a large number of tools, such as "gitlab-mcp", which contains 100+ tools
3. Copilot will collapse the tools according to their type, generating several virtual tools named "activate_type_tools"
4. The names and descriptions of the virtual tools are extremely inaccurate, and the tools are categorized in a very imprecise way. For example, the "list_issues" tool is categorized under "activate_group_and_repository_management_tools" instead of "activate_issue_management_tools"
5. The model is completely unable to find suitable tools on its own
6. The tool-search for custom endpoint models cannot be explicitly enabled, and virtual tools cannot be manually adjusted

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.