microsoft / microsoft/vscode

Do not derive model context from prompt and output token limits

Open
#332,995 0 comments 0 reactions 1 assignee Claimed by @lramos15 View on GitHub
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

- Copilot Chat Extension Version: Current
- VS Code Version: Current
- OS Version: N/A
- Feature (e.g. agent/edit/ask mode): Chat model picker and context usage UI
- Selected model (e.g. GPT 4.1, Claude 3.7 Sonnet): N/A (this can happen to any model that has input and output token limits that exceed the max context window limit)
- Logs: N/A

Steps to Reproduce:

1. Return synthetic model metadata with `max_prompt_tokens: 100000`, `max_output_tokens: 20000`, and `max_context_window_tokens: 100000`.
2. Open the model picker hover or context usage widget.
3. Observe that "Max context" can be calculated as 120,000 tokens rather than the declared 100,000-token context window.

Expected behavior:

Use `max_context_window_tokens` as the displayed total context window. Treat prompt and output limits as independent maxima rather than values whose sum defines context.

Actual behavior:

The model picker hover and context usage widget calculate total context as `maxInputTokens + maxOutputTokens`.

Relevant code:

- `src/vs/workbench/contrib/chat/browser/widget/input/modelPicker/modelPickerHover.ts`
- `src/vs/workbench/contrib/chat/browser/widgetHosts/viewPane/chatContextUsageWidget.ts`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.