microsoft / microsoft/vscode

Support Gemini 3.8 and future unlisted models via fallback capabilities and custom model settings

Open
#334,321 1 comment 3 reactions 1 assignee Claimed by @vijayupadya View on GitHub
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

## Summary
Currently, `GeminiNativeBYOKLMProvider` strictly filters out all models returned by the Google GenAI SDK that are not pre-registered in `this._knownModels`. When Google releases a new model or preview checkpoint (e.g., `gemini-2.5-*`, `gemini-3.7-flash`), users cannot select or use the model until Microsoft updates the remote `copilotChat.json` CDN manifest or ships a patch release.

We propose:
1. Adding a default capabilities fallback in `geminiNativeProvider.ts` (matching the pattern already used in `anthropicProvider.ts`).
2. Adding an optional setting in `settings.json` to allow user-defined Gemini model IDs and capability overrides.

---

## Current Behavior & Code Analysis

In `extensions/copilot/src/extension/byok/vscode-node/geminiNativeProvider.ts`, `getAllModels()` queries Google's API via `client.models.list()`, but discards any model not found in `_knownModels`:

```typescript
// extensions/copilot/src/extension/byok/vscode-node/geminiNativeProvider.ts
const client = new GoogleGenAI({ apiKey });
const models = await client.models.list();
const modelList: Record = {};

for await (const model of models) {
const modelId = model.name;
if (!modelId) {
continue;
}

// Strict filter: drops all new / unlisted models
if (this._knownModels && this._knownModels[modelId]) {
modelList[modelId] = this._knownModels[modelId];
}
}
return byokKnownModelsToAPIInfoWithEffort(this._name, modelList);
```

### Inconsistency with Other Providers
In `extensions/copilot/src/extension/byok/vscode-node/anthropicProvider.ts`, unlisted models returned by the API are not discarded; they automatically fall back to baseline capabilities:

```typescript
// extensions/copilot/src/extension/byok/vscode-node/anthropicProvider.ts
for (const model of response.data) {
if (this._knownModels && this._knownModels[model.id]) {
modelList[model.id] = this._knownModels[model.id];
} else {
// Mix in generic capabilities for models we don't know
modelList[model.id] = {
maxInputTokens: 100000,
maxOutputTokens: 16000,
name: model.display_name,
toolCalling: true,
vision: false,
thinking: false
};
}
}
```

---

## Proposed Solution

### 1. Capability Fallback for Discovered Gemini Models
Update `geminiNativeProvider.ts` to assign standard defaults to models returned by `client.models.list()` when they are missing from `_knownModels`:

```typescript
for await (const model of models) {
const modelId = model.name;
if (!modelId) {
continue;
}

if (this._knownModels && this._knownModels[modelId]) {
modelList[modelId] = this._knownModels[modelId];
} else {
// Fallback for newly released or uncatalogued models
modelList[modelId] = {
name: model.displayName || modelId,
maxInputTokens: 128000,
maxOutputTokens: 8192,
toolCalling: true,
vision: true,
supportsReasoningEffort: modelId.includes('thinking') || modelId.includes('3') ? ['low', 'medium', 'high'] : undefined,
};
}
}
```

### 2. User-Defined Custom Model Configuration (`settings.json`)
Allow users to manually register custom/fine-tuned/preview model IDs without waiting for API discovery or CDN manifest updates:

```json
{
"github.copilot.chat.byok.gemini.customModels": [
{
"id": "models/gemini-3.7-flash",
"name": "Gemini 3.7 Flash",
"maxInputTokens": 1000000,
"maxOutputTokens": 65536,
"toolCalling": true,
"vision": true
}
]
}
```

---

## Benefits
- **Zero-Day Availability:** Developers can use newly announced Gemini model IDs immediately upon release without waiting for extension or CDN updates.
- **Parity with Existing Providers:** Aligns the Gemini BYOK provider behavior with `AnthropicLMProvider` and `CustomOAIBYOKModelProvider`.
- **Custom / Fine-Tuned Model Support:** Enables Vertex AI / Google AI Studio custom-tuned endpoints and preview versions that may not appear in default manifest lists.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.