microsoft / microsoft/vscode

Flexible Context Window & Custom Thinking Effort for BYOK

Open
#331,011 0 comments 2 reactions 1 assignee Claimed by @vritant24 View on GitHub
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

*Additional part of #325237*

Add these 2 new features for Custom Endpoint:

1. Add flexible context window.

Add `availableContextWindow`:

```jsonc
{
"id": "custom",
// ...
"maxInputTokens": 1024000,
"maxOutputTokens": 64000, // `maxTokens` - `maxInputTokens` or 16k if not defined
"maxTokens": 1024000,
"availableContextWindow": [200000, 500000, 1000000] // Use the smallest CWS by default to reduce token consumption.
}
```
> [!NOTE]
> When both `availableContextWindow` and `maxInputTokens`/`maxTokens` are defined, use the former first.

2. Add custom thinking effort.

Add `effort`:

```jsonc
{
"id": "custom",
// ...
"efforts": ["off", "low", "medium", "high", "xhigh", "max", "minimal"]
// Display name: Off, Low, Medium, High, **Extra High**, Max, Minimal (usually for Gemini). Other efforts will be shown as-is.
"defaultEffort": "medium" // Default: Closest to `medium`. It cannot be `max`.
"useLegacyThinking": false // For older models which do not support efforts. Conflicts with `efforts` and `defaultEffort`.
}
```

3. Automatically save user's configuration of the settings above, except Max Effort.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.