Agents Window: BYOK models usable as main model but rejected for subagents (Copilot CLI model catalog omits bridged BYOK models)
- Dominant language
- TypeScript
- Stars
- 193k
- Forks
- 42.4k
- PR merge metrics
- PR metrics pending
Description
**Type: Bug**
## Summary
Extension-provided BYOK models (third-party `LanguageModelChatProvider`) work as the **main** model in an Agents Window session, but cannot be selected for a **subagent**. The agent advertises the model id as available, then the model is rejected at spawn time with `Model '' is not available` — while the very same error message lists that id under "Available models".
Agent-host logs show why: BYOK models are bridged into the session via a loopback proxy, but the Copilot CLI model catalog used for subagent model resolution contains **only CAPI models**.
## Environment
- OS: Windows 11 (Windows_NT), local workspace (no remote)
- Agent host / Agents Window session with Copilot harness
- `chat.agentHost.enabled: true`, `chat.agentHost.byokModels.enabled: true`
- BYOK provider: third-party extension `System-Sciences.vllm-copilot` (extension-provided `LanguageModelChatProvider`, `enabledApiProposals: chatProvider`)
- Model under test: `vllm-copilot/Qwen/Qwen3.8-Flash-Next-FP8 on ` (the picker id embeds the per-model `serverUrl`; hostnames redacted throughout)
## Steps to Reproduce
1. Register a third-party language-model provider extension whose models declare `capabilities.toolCalling: true` (and are `isBYOK: true`, user-selectable, no `targetChatSessionType`).
2. Enable `chat.agentHost.enabled` and `chat.agentHost.byokModels.enabled`, restart VS Code.
3. Open an Agents Window session on a local workspace, select the BYOK model in the model picker → **works**, the session runs on the model.
4. From that session, spawn a subagent / child task pinned to the same model id.
5. Observe rejection. Also try spawning a new delegated session with the same model id.
## Actual
**(a) Subagent spawn is rejected, with a self-contradicting message:**
```text
Model 'vllm-copilot/Qwen/Qwen3.8-Flash-Next-FP8 on ' is not available.
Available models: claude-sonnet-5, ...,
vllm-copilot/zai-org/GLM-5.2-FP8 on ,
vllm-copilot/deepseek/deepseek-v4-flash-vision-exp on openrouter.ai,
vllm-copilot/Qwen/Qwen3.8-Flash-Next-FP8 on
```
The rejected id is present in the list the error prints. Same for the qualified name form `Qwen3.8-Flash-Next (vllm-copilot)`, and for every other model from the same provider, so this is not model-specific.
**(b) Delegated sessions accept the model id but silently fall back:** creating a child session with the BYOK model id succeeds (no error), and the child runs on the default CAPI model instead. Silent substitution is arguably worse than an error, because the caller has no way to know which model actually executed.
**(c) Built-in models work** (`task` with e.g. `gemini-3.5-flash` runs normally), so the subagent mechanism itself is fine — only the third-party/BYOK path is broken.
## Root cause evidence
Agent host log (`logs//agenthost.log`):
```text
16:37:39 [Copilot] Wired 5 BYOK model(s) across 1 provider(s) via loopback proxy http://127.0.0.1:51787
16:39:01 [Copilot] Listing models...
16:39:01 [Copilot] Found 22 models: Auto, Claude Sonnet 5, Claude Sonnet 4.6, ..., Raptor mini
18:00:52 [Copilot] Changing model to: vllm-copilot/Qwen/Qwen3.8-Flash-Next-FP8 on
```
The BYOK bridge reports 5 models wired through `ByokLmProxyService`, and the main session switches to them successfully — but `Listing models...` for the Copilot CLI returns 22 models, all CAPI, none from the provider. There appear to be two model catalogs:
1. renderer/agent-host catalog including the bridged BYOK models (used by the model picker and the main session), and
2. the Copilot CLI catalog (used for subagent/child-task model resolution) which never receives the BYOK models.
This also explains the contradictory error: the advertised list comes from catalog 1, the validation from catalog 2.
## Why this is not a provider metadata problem
The current `runSubagentTool.ts` filter is:
```ts
ILanguageModelChatMetadata.suitableForAgentMode(m)
&& m.metadata.isUserSelectable !== false
&& !m.metadata.targetChatSessionType
```
with
```ts
suitableForAgentMode = (typeof capabilities?.agentMode === 'undefined' || capabilities.agentMode)
&& !!capabilities?.toolCalling
```
and `extHostLanguageModels.ts` derives `agentMode: !!m.capabilities.toolCalling` for extension providers automatically. The provider under test publishes `toolCalling: true`, omits `isUserSelectable` and `targetChatSessionType`, and is flagged `isBYOK: true` — it satisfies every predicate. `chat.agentHost.byokModels.enabled` is already enabled and demonstrably effective for the main session.
## Expected
A BYOK model that is selectable and runnable as the main Agents Window model should also be resolvable as a subagent/child-task model. If a BYOK model genuinely cannot serve a subagent, the id must not be advertised as available, and delegated-session creation with an unsupported model id must fail loudly instead of silently running a different model.
## Related
- https://github.com/microsoft/vscode/issues/315032 — "Subagent fails to resolve custom OAI model although available for main agent" (same symptom; this report adds the two-catalog root cause from agent-host logs, for extension-provided providers)
- https://github.com/microsoft/vscode/issues/325425 — subagents fail with Ollama BYOK models
- https://github.com/microsoft/vscode/issues/325738 — BYOK parity gaps in the Agents Window
Contributor guide
Assessment
This issue has not been assessed yet.