Copilot Chat: switching a session to autopilot mid-conversation injects task_complete instructions into the middle of the system prompt, invalidating prefix cache for third-party (chatProvider) models
- Dominant language
- TypeScript
- Stars
- 193k
- Forks
- 42.4k
- PR merge metrics
- PR metrics pending
Description
### Summary
When a chat session switches its permission level to **autopilot** mid-conversation, Copilot Chat injects the autopilot `task_complete` instruction **into the middle of the single system message**. For custom language model providers (via the `chatProvider` proposed API) that rely on automatic **prefix caching** (DeepSeek and other OpenAI-compatible endpoints, which have no cache-breakpoint concept), every token after the injection point misses the cache and the full turn is billed at full price.
The extension (`oai-compatible-copilot` style third-party providers) only forwards `messages` as received — it cannot reorder or split the merged system message — so this is a host-side rendering concern.
### Reproduction
1. Start an agent-mode chat in VS Code (default/assisted permission level).
2. Run a few turns so the backend has a large cached prefix (e.g. ~360k input tokens, >99% cache hits).
3. Mid-session, switch the conversation to **autopilot** (auto-approve) and send the next request.
4. Observe the upstream `cached_tokens` drop from ~99.9% of input to ~1%.
Both requests in step 2 and 3 carry the **same routing session** (same backend pod); only the request payload changes.
### Evidence from two captured requests (same conversation, 6 minutes apart)
| field | before (interactive) | after (switched to autopilot) |
|---|---|---|
| tools count | 80 (no `task_complete`) | 81 (**+ `task_complete`**) |
| system contains autopilot instruction | no | **yes, inserted at char 20,407** (306 chars) |
| upstream `usage.input_tokens` | 359,410 | 325,827 |
| upstream `cached_tokens` | **359,296 (99.9%)** | **3,328 (~1%)** |
The exact inserted text is the autopilot block:
> When you have fully completed the task, call the task_complete tool to signal that you are done.
> IMPORTANT: Before calling task_complete, you MUST provide a brief text summary of what was accomplished in your message. The task is not complete until both the summary and the task_complete call are present.
### Root cause (source pointers)
1. **`extensions/copilot/src/extension/intents/node/agentIntent.ts:289`** — `task_complete` is only enabled when `request.permissionLevel === 'autopilot'`.
2. **`extensions/copilot/src/extension/prompts/node/agent/agentPrompt.tsx:130, 140-143`** — the same flag gates a `` block that is rendered as part of `baseInstructions`, i.e. *inside the system section*, between the static identity/custom-instruction blocks and the user context:
```tsx
const isAutopilot = this.props.promptContext.request?.permissionLevel === 'autopilot';
...
{isAutopilot &&
When you have fully completed the task, call the task_complete tool to signal that you are done.
IMPORTANT: Before calling task_complete, you MUST provide a brief text summary of what was accomplished in your message. The task is not complete until both the summary and the task_complete call are present.
}
```
3. **`extensions/copilot/src/extension/prompts/node/base/promptRenderer.ts:107-131`** — consecutive system messages are collapsed into the single `messages[0]` text by concatenation, so that conditional block ends up embedded in the *middle* of the system prompt. On the observed request it landed at character ~20,407 of ~21,169, with the rest of the system text and the whole conversation history following after it.
4. Notably, the host already implements a protection against exactly this class of problem but does not cover these conditional blocks — **`agentPrompt.tsx:242-299` (`getOrFreezeCustomizationsIndex`)** snapshots the ``/``/`` customizations index on the first turn and reuses it, moving per-turn changes into a `drift` block rendered in the *latest user message*, explicitly "to stop per-turn churn … from invalidating the system prompt cache". The `isAutopilot` / `isVoiceModeInput` / `templateVariablesContext` conditional system blocks are outside that mechanism.
### Suggested fix
Render request-scoped, potentially *changing* conditional instructions (autopilot `task_complete`, voice-mode, template variables) the same way `drift` is handled — append them to the **latest user message** (or a trailing system block after conversation history) rather than merging them into the middle of the leading system prompt. Since these blocks only toggle when the session mode changes (entering/exiting autopilot), doing so keeps the system prefix byte-stable across turns and restores prefix-cache hits for OpenAI-compatible providers that lack `cache_control` breakpoints.
Alternative/complementary: when the conditional block *does* change, accept the one-time cache miss but keep the change at the very tail of the prompt (after history), limiting invalidation to the appended delta instead of the entire 300k+ token history.
### Environment
- VS Code: 1.136.0 (extension host forwards host-assembled `messages` unchanged to third-party providers via `chatProvider` proposed API)
- Copilot Chat: built-in (lockstep release)
- Model endpoint: DeepSeek v4 flash via an OpenAI-compatible gateway (`oai-compatible-copilot/0.4.2` extension acting as `LanguageModelChatProvider`)
- All observations made 2026-09-04
Contributor guide
Assessment
This issue has not been assessed yet.