microsoft / microsoft/vscode

BYOK OpenRouter: hover shows Max context 1M instead of 256K for ":free" variants (discovery prefers top-level context_length over top_provider.context_length)

Open
#332,917 3 comments 0 reactions 1 assignee Claimed by @vritant24 View on GitHub
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

- Copilot Chat Extension Version: GitHub.copilot-chat 0.63.0 (bundled with 1.135.0); 0.62.0 (1.134.0) when first reported
- VS Code Version: 1.135.0 — still reproduces; original report from 1.134.0
- OS Version: Windows 11 Pro for Workstations (10.0.26200), 64-bit
- Feature (e.g. agent/edit/ask mode): Chat model picker — BYOK (Manage Language Models) / OpenRouter provider
- Selected model (e.g. GPT 4.1, Claude 3.7 Sonnet): Poolside: Laguna S 2.1 (free) via OpenRouter BYOK group (`vendor: "openrouter"`)
- Logs: Discovery works without errors, nothing relevant in Output → "GitHub Copilot Chat". The root cause is reproducible with two public unauthenticated HTTP calls (see Root cause). Can attach `Developer: GitHub Copilot Chat Diagnostics` output on request.

Steps to Reproduce:

1. Model picker → ⚙️ Manage Language Models → Add Models → OpenRouter → enter API key, create a group.
2. Hover the model "Poolside: Laguna S 2.1 (free)" in the picker.
The hover card shows: **Max context 1M**.
3. Compare with https://openrouter.ai/poolside/laguna-s-2.1:free — the documented limit for this route is **256K** (262144 tokens).
Note: the paid sibling `poolside/laguna-s-2.1` shows its correct 1M, so the bug only affects variants whose real cap differs from the parent model.

## Root cause

The built-in OpenRouter provider discovers models via:

GET https://openrouter.ai/api/v1/models?supported_parameters=tools

and resolves capabilities approximately as (from the bundled minified provider):

```js
const a = entry.context_length ?? entry.top_provider.context_length; // ← prefers top-level
const c = Math.min(entry.top_provider.max_completion_tokens ?? 16000, Math.floor(a / 2));
// maxInputTokens = a - c, maxOutputTokens = c
// hover "Max context" = maxInputTokens + maxOutputTokens = a
```

On this filtered endpoint OpenRouter reports the parent paid model's `context_length`
at top level, while the true free-route cap is only present under `top_provider`.
Since the top-level field wins via `??`, VS Code surfaces the parent's limit.

Measured 2026-08-27, stable across repeated calls, no auth required:

```text
$ curl -s 'https://openrouter.ai/api/v1/models?supported_parameters=tools' \
| jq '.data[] | select(.id=="poolside/laguna-s-2.1:free") | {context_length, top_provider}'
{
"context_length": 1048576, ← parent value (wrong for :free)
"top_provider": { "context_length": 262144, "max_completion_tokens": 32768 }
}

$ curl -s 'https://openrouter.ai/api/v1/models' \
| jq '.data[] | select(.id=="poolside/laguna-s-2.1:free") | .context_length'
262144 ← consistent
```

So `a = 1048576`, `c = 32768`, total = 1048576 → picker advertises "1M".

Upstream: the same catalog inconsistency has been reported to OpenRouter (official Discord, `#feedback-and-requests`, 2026-08-27): https://discord.com/channels/1091220969173028894/1542502332435267604 — link resolves for OpenRouter Discord members only; the post contains the same repro and links back to this issue.

## Impact

The context-usage indicator and prompt truncation are computed from
`maxInputTokens + maxOutputTokens`. Requests above 262,144 tokens fail against
this free route even though the picker advertises 1M.

## Suggested fix

Prefer the smallest known limit instead of relying on field order:

```js
const ctx = Math.min(
entry.context_length ?? Infinity,
entry.top_provider?.context_length ?? entry.context_length
);
```

This stays correct for all current vendor response shapes (OpenRouter, Anthropic, xAI).

## Workaround

Re-declare the affected models through the Custom Endpoint provider with explicit
`maxInputTokens` / `maxOutputTokens` (e.g. 229376 / 32768 for this model) — then the
picker shows the correct window regardless of catalog data.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.