"Reserved for response" ignores `maxOutputTokens` for extension-contributed models (hard-coded to 8192)

Open
#321,761 1 comment 0 reactions 1 assignee View on GitHub

@vritant24 is already working on this.

Since Jun 17, 2026.

Assessment

This issue has not been assessed yet.

Description

bug model-byok

Problem

The Copilot context window widget shows "Reserved for response" as ~8192 tokens for all extension-contributed models, regardless of the maxOutputTokens value declared by the provider.

Expected: "Reserved for response" reflects the actual maxOutputTokens reported by the LanguageModelChatProvider (e.g., 32768, 65536, etc.).

Actual: Always ~8192 tokens, making the reservation bar a tiny sliver even for models configured with large output budgets.

Root Cause

In the Copilot Chat extension's internal endpoint wrapper, ExtensionContributedChatEndpoint ignores the provider's declared maxOutputTokens and returns a hard-coded value:

File: extensions/copilot/src/platform/endpoint/vscode-node/extChatEndpoint.ts

get maxOutputTokens(): number {
  // The VS Code API doesn't expose max output tokens, use a reasonable default
  return 8192;
}

This value flows into toolCallingLoop.ts where it's used as the outputBuffer for usage reporting:

stream.usage({
  completionTokens: fetchResult.usage.completion_tokens,
  promptTokens: fetchResult.usage.prompt_tokens,
  outputBuffer: endpoint.maxOutputTokens,  // ← always 8192 for extension models
  promptTokenDetails,
});

Which then renders in chatContextUsageWidget.ts as the "Reserved for response" bar.

Why This Matters

  • Extension providers do report maxOutputTokens on LanguageModelChatInformation — but Copilot's internal wrapper never reads it
  • For models with large output budgets (32K–128K), the UI misleads users into thinking only ~8K is reserved
  • This is particularly misleading for reasoning/thinking models where output can legitimately be very large
  • The maxOutputTokens value is used correctly for the actual vLLM max_tokens parameter — only the display is wrong

Context

We've already solved the related issues in our extension (vLLM-2-Copilot):

  • ✅ Token usage reporting via LanguageModelDataPart with 'usage' MIME type and snake_case keys (not blocked by isApiUsage())
  • prompt_tokens / completion_tokens / total_tokens displayed correctly in the context widget
  • provideTokenCount works for prompt budgeting

The only remaining gap is the outputBuffer / "Reserved for response" display.

Proposed Fix

Option A (preferred): Read maxOutputTokens from the model information when constructing ExtensionContributedChatEndpoint:

get maxOutputTokens(): number {
  return this.modelInfo.maxOutputTokens ?? 8192;
}

Option B: Expose maxOutputTokens on the LanguageModelChat API surface so Copilot can access it without reaching into provider internals.

Environment

Screenshot

Context Window Widget
Shows "Reserved for response" as a tiny bar (~8192) despite maxOutputTokens: 32768 being configured and correctly reported by the provider.


Related: This is a distinct issue from #314722 (usage = 0 for extension providers), which has already been addressed. The token counts (prompt, completion, total) display correctly — only the output reservation bar is affected.

Dominant language
TypeScript
Stars
193k
Forks
42.9k
PR merge metrics
PR metrics pending

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microsoft/vscode

All issues in microsoft/vscode

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.