google-gemini / google-gemini/gemini-cli

ACP: PromptResponse usage omits cached/thought tokens, inflating client cost estimates

Open
#27,985 5 comments 0 reactions 0 assignees View on GitHub
area/non-interactive effort/small status/bot-triaged
Dominant language
TypeScript
Stars
107k
Forks
14.6k
Avg merge
2d 3h
Merged PRs (30d)
45

Description

### What

When running as an ACP server (`gemini --acp`), the token usage reported for a prompt turn only includes **input** and **output** tokens. The **cached** and **thought/reasoning** token counts are dropped — even though Gemini's `usageMetadata` provides them and the CLI already consumes them elsewhere (telemetry, UI, chat recording).

In `packages/cli/src/acp/acpSession.ts`, the `GeminiEventType.Finished` handler reads only `promptTokenCount` and `candidatesTokenCount`:

```ts
const usage = event.value.usageMetadata;
if (usage) {
turnInputTokens = usage.promptTokenCount ?? turnInputTokens;
turnOutputTokens = usage.candidatesTokenCount ?? turnOutputTokens;
}
```

`cachedContentTokenCount` and `thoughtsTokenCount` are never read, and the emitted `_meta.quota.token_count` contains only `input_tokens` / `output_tokens`. By contrast, these fields *are* read in `packages/core/src/agent/event-translator.ts`, `packages/core/src/telemetry/types.ts`, and `packages/core/src/services/chatRecordingService.ts` — so the data is available; only the ACP path discards it.

### Why it matters

ACP clients that estimate cost from token counts treat **all** input tokens as uncached, because the cached count is absent/`0`. For long agentic sessions where most of the prompt is cached turn-to-turn, this **overstates the estimated cost by ~3×** relative to the real (cached) spend.

We hit this analyzing OpenHands eval runs through the Gemini CLI ACP harness: every instance reports `cache_read_tokens = 0`, so the token-based cost estimate is ~3× the actual proxy bill (the underlying spend already benefits from caching — only the *reporting* is wrong).

### Relationship to #24280

This is related to but distinct from #24280 (“populate the standard `PromptResponse.usage` field”). That issue's proposed payload lists only `input_tokens` / `output_tokens` / `total_tokens`, so implementing it as written would **not** close the cache-token gap. The standard ACP `Usage` type already has `cachedReadTokens` and `thoughtTokens` fields — they just need to be populated.

### Proposed fix

In `acpSession.ts`, also capture `usageMetadata.cachedContentTokenCount` and `usageMetadata.thoughtsTokenCount`, and populate the standard ACP `PromptResponse.usage` field (`inputTokens`, `outputTokens`, `cachedReadTokens`, `thoughtTokens`, `totalTokens`). This aligns Gemini CLI with Claude Agent ACP / Codex ACP and lets any ACP client read accurate usage without Gemini-specific `_meta.quota` parsing. The existing `_meta.quota` payload can be kept for backward compatibility.

I have a small PR ready for this.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.