Support Gemini 3.8 and future unlisted models via fallback capabilities and custom model settings
- Dominant language
- TypeScript
- Stars
- 193k
- Forks
- 42.4k
- PR merge metrics
- PR metrics pending
Description
## Summary
Currently, `GeminiNativeBYOKLMProvider` strictly filters out all models returned by the Google GenAI SDK that are not pre-registered in `this._knownModels`. When Google releases a new model or preview checkpoint (e.g., `gemini-2.5-*`, `gemini-3.7-flash`), users cannot select or use the model until Microsoft updates the remote `copilotChat.json` CDN manifest or ships a patch release.
We propose:
1. Adding a default capabilities fallback in `geminiNativeProvider.ts` (matching the pattern already used in `anthropicProvider.ts`).
2. Adding an optional setting in `settings.json` to allow user-defined Gemini model IDs and capability overrides.
---
## Current Behavior & Code Analysis
In `extensions/copilot/src/extension/byok/vscode-node/geminiNativeProvider.ts`, `getAllModels()` queries Google's API via `client.models.list()`, but discards any model not found in `_knownModels`:
```typescript
// extensions/copilot/src/extension/byok/vscode-node/geminiNativeProvider.ts
const client = new GoogleGenAI({ apiKey });
const models = await client.models.list();
const modelList: Record = {};
for await (const model of models) {
const modelId = model.name;
if (!modelId) {
continue;
}
// Strict filter: drops all new / unlisted models
if (this._knownModels && this._knownModels[modelId]) {
modelList[modelId] = this._knownModels[modelId];
}
}
return byokKnownModelsToAPIInfoWithEffort(this._name, modelList);
```
### Inconsistency with Other Providers
In `extensions/copilot/src/extension/byok/vscode-node/anthropicProvider.ts`, unlisted models returned by the API are not discarded; they automatically fall back to baseline capabilities:
```typescript
// extensions/copilot/src/extension/byok/vscode-node/anthropicProvider.ts
for (const model of response.data) {
if (this._knownModels && this._knownModels[model.id]) {
modelList[model.id] = this._knownModels[model.id];
} else {
// Mix in generic capabilities for models we don't know
modelList[model.id] = {
maxInputTokens: 100000,
maxOutputTokens: 16000,
name: model.display_name,
toolCalling: true,
vision: false,
thinking: false
};
}
}
```
---
## Proposed Solution
### 1. Capability Fallback for Discovered Gemini Models
Update `geminiNativeProvider.ts` to assign standard defaults to models returned by `client.models.list()` when they are missing from `_knownModels`:
```typescript
for await (const model of models) {
const modelId = model.name;
if (!modelId) {
continue;
}
if (this._knownModels && this._knownModels[modelId]) {
modelList[modelId] = this._knownModels[modelId];
} else {
// Fallback for newly released or uncatalogued models
modelList[modelId] = {
name: model.displayName || modelId,
maxInputTokens: 128000,
maxOutputTokens: 8192,
toolCalling: true,
vision: true,
supportsReasoningEffort: modelId.includes('thinking') || modelId.includes('3') ? ['low', 'medium', 'high'] : undefined,
};
}
}
```
### 2. User-Defined Custom Model Configuration (`settings.json`)
Allow users to manually register custom/fine-tuned/preview model IDs without waiting for API discovery or CDN manifest updates:
```json
{
"github.copilot.chat.byok.gemini.customModels": [
{
"id": "models/gemini-3.7-flash",
"name": "Gemini 3.7 Flash",
"maxInputTokens": 1000000,
"maxOutputTokens": 65536,
"toolCalling": true,
"vision": true
}
]
}
```
---
## Benefits
- **Zero-Day Availability:** Developers can use newly announced Gemini model IDs immediately upon release without waiting for extension or CDN updates.
- **Parity with Existing Providers:** Aligns the Gemini BYOK provider behavior with `AnthropicLMProvider` and `CustomOAIBYOKModelProvider`.
- **Custom / Fine-Tuned Model Support:** Enables Vertex AI / Google AI Studio custom-tuned endpoints and preview versions that may not appear in default manifest lists.
Contributor guide
Assessment
This issue has not been assessed yet.