AgentOps-AI / AgentOps-AI/agentops
Anthropic prompt caching: cache tokens not emitted on the Anthropic paths — is session total_tokens systematically low?
- Vorherrschende Sprache
- Python
- Sterne
- 5.8k
- Forks
- 619
- PR-Merge-Kennzahlen
- Keine gemergten PRs in 30 T.
Beschreibung
A semantics question found while auditing token accounting across agent-telemetry SDKs.
**The observation**
On the Anthropic paths, usage extraction reads only `usage.input_tokens` / `usage.output_tokens` and computes `total = input + output`:
- non-streaming: `agentops/instrumentation/providers/anthropic/attributes/message.py` (~L319-332)
- streaming: `.../anthropic/stream_wrapper.py` (~L167-177 and ~L406-418)
Anthropic's `cache_read_input_tokens` / `cache_creation_input_tokens` are separate fields (not included in `input_tokens`), and I don't find them read anywhere on these paths — `LLM_USAGE_CACHE_READ_INPUT_TOKENS` / `CACHE_CREATION_INPUT_TOKENS` are defined in `agentops/semconv/span_attributes.py` but have no setter calls in the Anthropic instrumentation.
Meanwhile the OpenAI Responses path (`providers/openai/attributes/response.py`, ~L296-307) does emit `cached_tokens → cache_read` and `reasoning_tokens` — so the asymmetry is within one SDK.
**Why it matters**
The OTel GenAI semantic conventions for Anthropic (`semantic-conventions-genai`, `docs/gen-ai/anthropic.md`) are explicit: cache read/write tokens **MUST be added** to the native `input_tokens` to compute `gen_ai.usage.input_tokens`. If they're dropped instead, a prompt-caching Claude session reports `total_tokens` systematically below what the provider actually meters (cache reads are billed, just at a lower rate — they're not free tokens).
**The question**
Is this asymmetry intentional (e.g. a deliberate "native usage only" convention on the Anthropic path)? If not: for sessions with `cache_control` enabled, does any downstream token/cost figure derived from these spans undercount? Happy to send a minimal repro — same prompt called twice with caching on, asserting the cache attributes are absent and the total vs the Anthropic console's billed tokens diverge — or a small PR adding the cache fields per the semconv rule.
Context: I maintain AgentMeasure, a measurement-conformance project running a public 30-projects/30-days check of agent-telemetry semantics; this is the finding worth asking about directly. Not claiming a bug without your read on the intended convention.
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.