Agent-mode workspace directory tree is injected ahead of the cached prefix and breaks BYOK Anthropic prompt caching

Open
#323,641 2 comments 0 reactions 1 assignee View on GitHub

@roblourens is already working on this.

Since Jun 29, 2026.

Assessment

This issue has not been assessed yet.

Description

Type

Bug — performance/cost regression on the BYOK Anthropic path.

Environment
  • VS Code: 1.126.0 (commit 7e7950df89d055b5a378379db9ee14290772148a)
  • GitHub Copilot Chat: <your Extensions → GitHub Copilot Chat version>
  • OS: macOS (expected on all platforms)
  • Model: BYOK Anthropic (Claude Opus 4.8) via ANTHROPIC_BASE_URL
  • Occurs with all other extensions disabled: yes
Summary

In agent mode the prompt includes an injected workspace directory structure tree
("I am working in a workspace that has the following structure:") placed near the head of the
request, ahead of the large otherwise-static cached prefix (instructions / AGENTS.md / tool
definitions).

That tree is regenerated each turn and reflects the live workspace at depth ≤ 2. When any
directory at depth ≤ 2 of a workspace/repo root is created or removed mid-session (very common —
per-ticket scratch dirs, screenshot folders, test output, downloads), the tree text changes.
Because it sits before the cached prefix, the byte change invalidates the entire downstream
prefix, forcing a full Anthropic cache write (1.25× input rate) instead of a cache read
(0.1×).

There is no setting to disable or stabilize this injection.

Impact

On a real BYOK Anthropic workload with a ~270K-token cached prefix:

  • Measured reuse factor fell to ~4× (well-tuned Anthropic caching is 20–25×).
  • ~22% of turns were full-prefix re-writes, and 94% of those re-writes happened within 5
    minutes of the prior turn     (i.e. not TTL expiry — a structural bust).
  • Captured the exact moving byte: a single artifacts/p12-NNNN/ directory appearing/disappearing
    at byte ~800 of a ~669KB request body, cold-rewriting the ~270K-token prefix after it.
Steps to reproduce
  1. Use agent mode with a BYOK Anthropic model and a sizeable cached prefix (instructions +
    AGENTS.md + tools).
  2. Capture two consecutive raw /v1/messages request bodies (e.g. via a transparent local forward
    proxy on ANTHROPIC_BASE_URL).
  3. Between the two turns, create or remove a directory at depth ≤ 2 of a workspace root
    (e.g. mkdir artifacts/some-dir).
  4. Diff the two bodies: the first divergence is inside the "following structure:" tree near the
    head; cache_creation_input_tokens spikes and cache_read_input_tokens drops to 0 on the
    second turn.
Root cause (from the shipped extension, 1.126.0)

The tree is emitted by an unconditional PromptElement (minified class LP):

Dominant language
TypeScript
Stars
193k
Forks
42.9k
PR metrics pending

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microsoft/vscode

All issues in microsoft/vscode

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.