Support auto model selection in .agent.md based on task complexity
- Dominant language
- TypeScript
- Stars
- 193k
- Forks
- 42.4k
- PR merge metrics
- PR metrics pending
Description
## Summary
Allow custom agents () to declare a **model selection policy** instead of (or in addition to) a static model, so VS Code can pick the best available model for each invocation based on task complexity, context size, or tool requirements.
## Problem
Currently, custom agents pin a single model (or a static fallback array) in frontmatter:
This creates a hard trade-off:
| Scenario | Static model choice | Consequence |
|---|---|---|
| Simple refactor ("add a type annotation") | Expensive / large model | Wastes tokens and money |
| Complex architecture analysis | Cheap / small model | Poor quality, misses nuance |
| High-tool-calling task (CI, k8s manifests) | Model without reliable tool calling | Fails silently or hallucinates tool args |
| Long-context summarization | Small-context model | Truncates, loses critical info |
Multi-agent setups amplify this: an orchestrator dispatches to 5+ phase agents, each hardcoded to one model. The orchestrator cannot adapt model selection to the workload it's handing off.
## Proposed solution
Add a (or ) frontmatter field alongside :
### How selection would work
1. VS Code already knows each model's capabilities from (tool calling, vision, max tokens, context window).
2. At invocation time, the agent framework inspects the user's request + tool set + conversation context length.
3. It selects the cheapest model whose capabilities satisfy the policy constraints.
4. If no model qualifies, fall back to the explicit list or the field.
### Capability-aware selection signals
| Signal | Source | Effect |
|---|---|---|
| Tool calling required | Agent's list is non-empty | Filter to models with |
| Large context | Conversation history + attachments > 64K tokens | Filter to models with ≥ context size |
| Vision needed | User attached image(s) | Filter to models with |
| Task complexity | User prompt heuristic (code analysis, multi-file refactor, architecture) | Prefer larger models when quality-first; smaller when cost-optimal |
### Interaction with existing field
model
## Why this matters
- **Cost**: Users with BYOK endpoints (token-plan models, self-hosted) pay per token. Routing trivial tasks to a 1M-context model is wasteful.
- **Quality**: Complex tasks on small models degrade the agent experience silently — the agent works but produces shallow output.
- **Orchestrator workflows**: Multi-agent setups (DE → SE → EM pattern, or CI pipeline agents) need per-task model selection. The orchestrator knows the phase but can't express it in the current model field.
- **Provider diversity**: As more BYOK providers are added (LM Studio, cloud endpoints, token plans), the selection problem grows combinatorially.
## Current workaround
Static fallback arrays with manual ordering:
This just picks the first available — it doesn't adapt to task needs.
## Related
- [#319709](https://github.com/microsoft/vscode/issues/319709) — Custom endpoint model resolution format (now resolved: format)
- capability fields (, , )
- Custom agent field documentation
Contributor guide
Assessment
This issue has not been assessed yet.