AgentOps-AI / AgentOps-AI/agentops

Anthropic prompt caching: cache tokens not emitted on the Anthropic paths — is session total_tokens systematically low?

Abierto
#1,445 2 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
5.8k
Forks
619
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

A semantics question found while auditing token accounting across agent-telemetry SDKs.

**The observation**

On the Anthropic paths, usage extraction reads only `usage.input_tokens` / `usage.output_tokens` and computes `total = input + output`:

- non-streaming: `agentops/instrumentation/providers/anthropic/attributes/message.py` (~L319-332)
- streaming: `.../anthropic/stream_wrapper.py` (~L167-177 and ~L406-418)

Anthropic's `cache_read_input_tokens` / `cache_creation_input_tokens` are separate fields (not included in `input_tokens`), and I don't find them read anywhere on these paths — `LLM_USAGE_CACHE_READ_INPUT_TOKENS` / `CACHE_CREATION_INPUT_TOKENS` are defined in `agentops/semconv/span_attributes.py` but have no setter calls in the Anthropic instrumentation.

Meanwhile the OpenAI Responses path (`providers/openai/attributes/response.py`, ~L296-307) does emit `cached_tokens → cache_read` and `reasoning_tokens` — so the asymmetry is within one SDK.

**Why it matters**

The OTel GenAI semantic conventions for Anthropic (`semantic-conventions-genai`, `docs/gen-ai/anthropic.md`) are explicit: cache read/write tokens **MUST be added** to the native `input_tokens` to compute `gen_ai.usage.input_tokens`. If they're dropped instead, a prompt-caching Claude session reports `total_tokens` systematically below what the provider actually meters (cache reads are billed, just at a lower rate — they're not free tokens).

**The question**

Is this asymmetry intentional (e.g. a deliberate "native usage only" convention on the Anthropic path)? If not: for sessions with `cache_control` enabled, does any downstream token/cost figure derived from these spans undercount? Happy to send a minimal repro — same prompt called twice with caching on, asserting the cache attributes are absent and the total vs the Anthropic console's billed tokens diverge — or a small PR adding the cache fields per the semconv rule.

Context: I maintain AgentMeasure, a measurement-conformance project running a public 30-projects/30-days check of agent-telemetry semantics; this is the finding worth asking about directly. Not claiming a bug without your read on the intended convention.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.