microsoft / microsoft/vscode

GitHub Copilot thinking effort purportedly used does not reflect thinking effort requested.

Open
#325,447 1 comment 0 reactions 1 assignee Claimed by @justschen View on GitHub
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

1. Using Claude Sonnet 5, set the thinking effort to `Max`.
2. Submit Prompt A, such as `/foo`. Confirm that the new prompt field thinking effort is still set to `Max`.
3. **Edit the submitted Prompt A** (`/foo`) and set its thinking effort to `Low`; then **resubmit Prompt A**. Confirm that the new prompt field thinking effort is still set to `Max`.
4. Submit Prompt B, such as `/bar`. Because the new prompt field thinking effort is still set to `Max`, the user expects Prompt P to be be submitted at thinking effort `Max`.
5. Edit the submitted Prompt B and observe the thinking effort shown for that submitted prompt.

**Expected:** The thinking effort of the submitted Prompt B is `Max`, because that is what was set when this new prompt was submitted.

**Actual:** The thinking effort of the submitted Prompt B actually shows `Low`.

**Discussion:** There is a bug one way or another: either

1. The prompt was submitted with the wrong thinking effort level, or
2. The prompt was submitted with the requested thinking effort level, but the submitted prompt does not retain what thinking level was used and shows another thinking level.

Either way this is a big problem for users. If I submit a prompt with one thinking effort level but VS Code does not honor that level, then I am not getting what I paid for and what I requested. If it did honor that request but then shows the wrong information, the user is left not knowing at what thinking effort the prompts are being submitted.

**Compounding Issue:** GitHub Copilot shows the model and the number of credits used for each turn, but does not show what parameters were used. Moreover I looked at the debug output, and I don't see any indication of which thinking effort was actually used! It only shows `"model": "claude-sonnet-5"`, but with no indication of thinking effort. Thus as it stands, _I can't even know if a workaround would work, because it's not clear which indication is correct: the thinking level I requested, or the thinking level shown when I edit the prompt_.

**Workaround:** (unsatisfactory; unsure if it even works)

1. Submit a prompt at the thinking effort level requested.
2. Edit and resubmit the prompt to ensure it shows the requested level, and if not resubmit it at the appropriate level.
3. Never change the thinking effort level again for that model.

**Summary:** I have no idea what thinking level is actually being used, but it is undeniable that there is a bug somewhere.

**Related Tickets:** Note that this is similar erroneous functionality to bug #319743. You might check whether the techniques used to keep track of models and their parameters for submitted prompts and their settings for resubmission is brittle and/or under-tested.

```
Version: 1.128.0 (user setup)
Commit: fc3def6774c76082adf699d366f31a557ce5573f
Date: 2026-07-07T15:14:24-07:00
Electron: 42.5.0
ElectronBuildId: 14525058
Chromium: 148.0.7778.271
Node.js: 24.17.0
V8: 14.8.178.33-electron.0
OS: Windows_NT x64 10.0.26200
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.