When using custom endpoints, virtual tools cause the model to fail to accurately discover the tools
- Dominant language
- TypeScript
- Stars
- 193k
- Forks
- 42.4k
- PR merge metrics
- PR metrics pending
Description
- Copilot Chat Extension Version: 1.0.81-0
- VS Code Version: 1.136.2
- OS Version: Windows 11 24H2
- Feature (e.g. agent/edit/ask mode): Tool call
- Selected model (e.g. GPT 4.1, Claude 3.7 Sonnet): Custom endpoint
Steps to Reproduce:
1. Use API Keys to access custom endpoint models(or OpenRouter), such as GPT 5.6 Sol
2. Connect to an MCP that includes a large number of tools, such as "gitlab-mcp", which contains 100+ tools
3. Copilot will collapse the tools according to their type, generating several virtual tools named "activate_type_tools"
4. The names and descriptions of the virtual tools are extremely inaccurate, and the tools are categorized in a very imprecise way. For example, the "list_issues" tool is categorized under "activate_group_and_repository_management_tools" instead of "activate_issue_management_tools"
5. The model is completely unable to find suitable tools on its own
6. The tool-search for custom endpoint models cannot be explicitly enabled, and virtual tools cannot be manually adjusted
Contributor guide
Assessment
This issue has not been assessed yet.