microsoft / microsoft/vscode

Agents Window over SSH: BYOK (custom model providers) unsupported + agent-host version mismatch blocks usage

Open
#325,738 8 comments 25 reactions 1 assignee Claimed by @vritant24 View on GitHub
feature-request model-byok
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

The Agents Window experience is very promising, but using it with SSH-based agent sessions is currently difficult for users who rely on BYOK/custom model providers.

There are three related problems:

1. Local workspaces can use a local agent/custom provider path, but SSH agent sessions cannot.
2. The SSH agent host can fail due to server/client protocol version mismatches.
3. Custom model provider extensions require manual allow-listing for the Agents Window.

Together, these make SSH-based Agents Window usage much harder than local usage, even though the underlying model/provider setup works outside the SSH Agents Window.

## 1. SSH Agent lacks BYOK parity with Local Agent
- ✅ Local Workspace: By setting `"sessions.chat.localAgent.enabled": true`, the Local Agent starts successfully and BYOK custom models work perfectly.
- ❌ SSH Agent Session: Fails to use BYOK models, no Local Agent option.

Root Cause (https://github.com/microsoft/vscode/issues/320339#issuecomment-4724977415):
The remote agent-host session is created without passing the BYOK provider. The copilotSessionLauncher.ts only sends the gitHubToken and never threads a custom provider/base-URL through to the SSH Agent session.

## 2. SSH Agent Connection Fails (Protocol Version Mismatch)

When opening the Agents Window over SSH, the agent-host connection can fail with a protocol mismatch:

> Failed to connect to SSH agent-host: Client offered protocol versions [0.5.1], none of which are compatible with this server's version 0.5.0 (server accepts ^0.5.0).

Using the built-in `Update server version` option does not resolve the issue. Deleting `~/.vscode-server` on remote host does not resolve the issue.

Root Cause (https://github.com/microsoft/vscode/issues/317380#issuecomment-4720225429):
The agent host resolves quality Stable against the public update channel and downloads the channel-latest server, instead of pinning to the client''s exact commit (the way Remote-SSH/WSL does).

## 3. High barrier for enabling custom model provider extensions

To use third-party model provider extensions in the Agents Window, users must manually add undocumented settings like:

```
"extensions.supportAgentsWindow": {
"SmallMain.vscode-unify-chat-provider": true,
"vicanent.gcmp": true,
"ltmoerdani.opencode-copilot-chat": true
}
```

This is a high barrier to entry, especially for users who simply want to use a custom API key through an extension. Simplifying this experience (or exposing a UI toggle) would greatly improve the out‑of‑box BYOK experience.

## Why this matters

The Agents Window is already a very promising experience, but remote SSH workflows and BYOK support are essential for many advanced users and teams.

Many developers run their real workloads on remote machines, and many organizations want the flexibility to bring their own model providers, endpoints, or API keys. If the Agents Window can support this workflow smoothly, VS Code would become a much stronger platform for agentic development, especially compared with other emerging alternatives like Orca.

Improving SSH + BYOK support would not only make the Agents Window more useful to existing VS Code users, but also help grow the broader Copilot ecosystem. A better BYOK story can reduce adoption friction, bring more users into the VS Code agent experience, and ultimately increase Copilot engagement as well.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.