AgentOps-AI / AgentOps-AI/agentops

Anthropic prompt caching: cache tokens not emitted on the Anthropic paths — is session total_tokens systematically low?

Ouverte
#1,445 2 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
5.8k
Forks
619
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

A semantics question found while auditing token accounting across agent-telemetry SDKs.

**The observation**

On the Anthropic paths, usage extraction reads only `usage.input_tokens` / `usage.output_tokens` and computes `total = input + output`:

- non-streaming: `agentops/instrumentation/providers/anthropic/attributes/message.py` (~L319-332)
- streaming: `.../anthropic/stream_wrapper.py` (~L167-177 and ~L406-418)

Anthropic's `cache_read_input_tokens` / `cache_creation_input_tokens` are separate fields (not included in `input_tokens`), and I don't find them read anywhere on these paths — `LLM_USAGE_CACHE_READ_INPUT_TOKENS` / `CACHE_CREATION_INPUT_TOKENS` are defined in `agentops/semconv/span_attributes.py` but have no setter calls in the Anthropic instrumentation.

Meanwhile the OpenAI Responses path (`providers/openai/attributes/response.py`, ~L296-307) does emit `cached_tokens → cache_read` and `reasoning_tokens` — so the asymmetry is within one SDK.

**Why it matters**

The OTel GenAI semantic conventions for Anthropic (`semantic-conventions-genai`, `docs/gen-ai/anthropic.md`) are explicit: cache read/write tokens **MUST be added** to the native `input_tokens` to compute `gen_ai.usage.input_tokens`. If they're dropped instead, a prompt-caching Claude session reports `total_tokens` systematically below what the provider actually meters (cache reads are billed, just at a lower rate — they're not free tokens).

**The question**

Is this asymmetry intentional (e.g. a deliberate "native usage only" convention on the Anthropic path)? If not: for sessions with `cache_control` enabled, does any downstream token/cost figure derived from these spans undercount? Happy to send a minimal repro — same prompt called twice with caching on, asserting the cache attributes are absent and the total vs the Anthropic console's billed tokens diverge — or a small PR adding the cache fields per the semconv rule.

Context: I maintain AgentMeasure, a measurement-conformance project running a public 30-projects/30-days check of agent-telemetry semantics; this is the finding worth asking about directly. Not claiming a bug without your read on the intended convention.

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.