microsoft / microsoft/vscode

Include cache creation (write) tokens in Copilot Chat debug-log main.jsonl llm_request events

Open
#329,657 0 comments 0 reactions 1 assignee Claimed by @zhichli View on GitHub
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

## Summary

The Copilot Chat debug-log `main.jsonl` `llm_request` events currently include `inputTokens`, `outputTokens`, and `cachedTokens` (cache read). However, **cache creation (write) tokens** are not recorded. This makes it impossible to accurately track the full token cost of models that support prompt caching (e.g., Anthropic Claude models), since cache creation tokens are billed at a different rate than standard input tokens.

## Current behavior

An `llm_request` entry in `main.jsonl` includes:

```json
{
"type": "llm_request",
"attrs": {
"model": "global.anthropic.claude-sonnet-5",
"inputTokens": 56261,
"outputTokens": 3762,
"cachedTokens": 52107,
...
}
}
```

`cachedTokens` captures cache **read** tokens (tokens served from the cache). However, there is no field for cache **creation** (write) tokens — the tokens written to the cache on a given request, which are typically billed at a premium (e.g., 1.25× the standard input rate for Anthropic models).

## Why this matters

For usage tracking and cost analysis tools that read these debug logs:

1. **Cache write tokens are billable.** Anthropic's API charges for cache creation at a higher rate than standard input tokens. Without this field, cost calculations underestimate actual spend.
2. **The API response already includes this data.** The underlying API response from Anthropic models returns `cache_creation_input_tokens` alongside `cache_read_input_tokens`. The information is available — it's just not being logged.
3. **`cachedTokens` alone is insufficient.** A request can both read from an existing cache and write new content to the cache. These are distinct token counts with distinct billing rates, and both need to be recorded for accurate accounting.

## Proposed change

Add a `cacheCreationTokens` (or `cacheWriteTokens`) field to the `attrs` object of `llm_request` events in `main.jsonl`:

```json
{
"type": "llm_request",
"attrs": {
"model": "global.anthropic.claude-sonnet-5",
"inputTokens": 56261,
"outputTokens": 3762,
"cachedTokens": 52107,
"cacheCreationTokens": 4154,
...
}
}
```

This would mirror the `cache_creation_input_tokens` / `cache_read_input_tokens` split that Anthropic's API already returns.

For models or providers that don't report cache creation tokens, the field can be omitted or set to `0` (consistent with how `cachedTokens` behaves today for models that don't report cache reads).

## Context

- VS Code version: 1.102.0 (Insiders)
- Copilot Chat extension version: 0.60.0
- Observed in: `%APPDATA%\Code - Insiders\User\workspaceStorage\\GitHub.copilot-chat\debug-logs\\main.jsonl`
- The `cachedTokens` field is present for Anthropic models but absent for other models (e.g., `zai-org/GLM-5.2-FP8`), which is expected since not all providers support prompt caching

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.