microsoft / microsoft/vscode

Charging for Failed Tool Calls

Open
#326,094 0 comments 0 reactions 1 assignee Claimed by @roblourens View on GitHub
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

- Copilot Chat Extension Version: All
- VS Code Version: All
- OS Version: All
- Feature (e.g. agent/edit/ask mode): Tool Calls
- Selected model (e.g. GPT 4.1, Claude 3.7 Sonnet): All
- Logs: All

Steps to Reproduce:

It really sucks that GitHub/VSCode is charging for failed tool calls. It seems this sets up a negative incentive for increasing the number of failed tools, which result in re-trying many times with either other tools, or with different input. This burns tokens at a very high rate...especially for big changes...

It would be good if:

1) we track and display to the user which tools are being called, and how often these calls fail
2) work to minimize these by fixing issues, and improving the tooling so that it fails less
3) optionally do not charge for tokens when there are significant numbers of tool failures (especially the default/built-in ones), or provide credits (maybe at 50%)
4) when tools are regularly failing, disable them, or look for alternatives to enable

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.