Feature Request: Support BYOK / Custom Models for Inline Completions (Ghost Text)
- Dominant language
- TypeScript
- Stars
- 193k
- Forks
- 42.4k
- PR merge metrics
- PR metrics pending
Description
## Summary
Currently, BYOK (Bring Your Own Key) and custom model providers only work for **Chat** (Panel, Inline Chat, Agent Mode) via the `LanguageModelChatProvider` API. **Inline completions (Ghost Text) are hardwired to GitHub's cloud infrastructure** and cannot use custom models.
This creates a critical gap for:
- **Air-gapped environments** (government, defense, critical infrastructure)
- **Data privacy compliance** (GDPR, HIPAA, SOC 2)
- **Cost optimization** (self-hosted models for high-volume completions)
- **Offline development** (no internet connection)
## Current Behavior
1. ✅ Chat works with BYOK models (Ollama, OpenAI-compatible, Anthropic, etc.)
2. ❌ Inline completions **require GitHub Copilot cloud** (no BYOK support)
3. ❌ Setting `github.copilot.selectedCompletionModel` to a custom model ID is **silently ignored**
4. ❌ Disconnecting from the internet **disables completions entirely**
## Proposed Solution
Extend the `LanguageModelChatProvider` API (or introduce a new `LanguageModelCompletionProvider` API) to support custom model providers for inline completions.
### Key Requirements
1. **New API Proposal** (or extend `chatProvider`):
- Add `ChatLocation.Completion` to the `ChatLocation` enum
- Allow `LanguageModelChatProvider` to serve completion requests
- OR: Create `registerLanguageModelCompletionProvider()` API
2. **Prompt Format Support**:
- Completions use a specialized format (prefix/suffix/blockMode/indentation)
- The API should abstract this, or provide a standard completion interface
- Consider OpenAI's `/v1/completions` format as a baseline
3. **Model Validation**:
- Remove the `genericModels` whitelist filter for BYOK providers
- Allow custom model IDs to bypass CAPI validation when registered via BYOK
4. **Configuration**:
- Extend `github.copilot.chat.customOAIModels` (or new setting) to include completion models
- Allow separate model selection for Chat vs. Completions
## Use Cases
### Air-Gapped Government Agency
```jsonc
// .vscode/settings.json
{
"github.copilot.chat.customOAIModels": [
{
"id": "local-coder",
"name": "Local CodeLlama 70B",
"baseUrl": "http://llm-server.internal:8080/v1",
"apiKey": "none",
"model": "codellama-70b",
"capabilities": {
"chat": true,
"completion": true // ← NEW: Support completions
}
}
]
}
```
### Cost-Optimized Enterprise
```jsonc
{
"github.copilot.chat.customOAIModels": [
{
"id": "deepseek-coder",
"name": "DeepSeek Coder (Self-hosted)",
"baseUrl": "http://ai-pool.internal:8080/v1",
"capabilities": {
"chat": false, // Use Copilot for chat (complex reasoning)
"completion": true // Use self-hosted for completions (high-volume)
}
}
]
}
```
## Acceptance Criteria
- [ ] Custom models can be registered for inline completions via API
- [ ] `ChatLocation` enum includes a `Completion` value (or equivalent)
- [ ] `genericModels` whitelist is bypassed for BYOK-registered completion models
- [ ] Completion prompt format is abstracted or documented for custom providers
- [ ] Configuration allows separate model selection for Chat vs. Completions
- [ ] Air-gapped environments can use completions without internet connectivity
- [ ] No regression for existing Copilot completion behavior
## Related Issues
- #318545 - Support BYOK/Custom Models for Inline Code Completions (156+ upvotes)
- #246551 - Use copilot with local models completely offline (229+ upvotes)
- #246303 - Add CoPilot configuration option for OpenAI compatible endpoints
## Environment
- VS Code Version: 1.104+
- OS: All (Windows, macOS, Linux)
- Copilot Extension: Latest
## Additional Context
The `LanguageModelChatProvider` API already supports BYOK for Chat locations (Agent, Terminal, Notebook, InlineChat). Extending this to Completions would be a natural evolution of the existing API, maintaining consistency and reducing implementation complexity.
The technical infrastructure exists — the `byok/` directory in the Copilot extension already contains implementations for Anthropic, Gemini, XAI, Azure, and custom OpenAI-compatible providers. These just need to be wired into the completions pipeline.
Contributor guide
Assessment
This issue has not been assessed yet.