AgentOps-AI / AgentOps-AI/agentops
Anthropic prompt caching: cache tokens not emitted on the Anthropic paths — is session total_tokens systematically low?
- Ngôn ngữ chính
- Python
- Star
- 5.8k
- Fork
- 619
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
A semantics question found while auditing token accounting across agent-telemetry SDKs.
**The observation**
On the Anthropic paths, usage extraction reads only `usage.input_tokens` / `usage.output_tokens` and computes `total = input + output`:
- non-streaming: `agentops/instrumentation/providers/anthropic/attributes/message.py` (~L319-332)
- streaming: `.../anthropic/stream_wrapper.py` (~L167-177 and ~L406-418)
Anthropic's `cache_read_input_tokens` / `cache_creation_input_tokens` are separate fields (not included in `input_tokens`), and I don't find them read anywhere on these paths — `LLM_USAGE_CACHE_READ_INPUT_TOKENS` / `CACHE_CREATION_INPUT_TOKENS` are defined in `agentops/semconv/span_attributes.py` but have no setter calls in the Anthropic instrumentation.
Meanwhile the OpenAI Responses path (`providers/openai/attributes/response.py`, ~L296-307) does emit `cached_tokens → cache_read` and `reasoning_tokens` — so the asymmetry is within one SDK.
**Why it matters**
The OTel GenAI semantic conventions for Anthropic (`semantic-conventions-genai`, `docs/gen-ai/anthropic.md`) are explicit: cache read/write tokens **MUST be added** to the native `input_tokens` to compute `gen_ai.usage.input_tokens`. If they're dropped instead, a prompt-caching Claude session reports `total_tokens` systematically below what the provider actually meters (cache reads are billed, just at a lower rate — they're not free tokens).
**The question**
Is this asymmetry intentional (e.g. a deliberate "native usage only" convention on the Anthropic path)? If not: for sessions with `cache_control` enabled, does any downstream token/cost figure derived from these spans undercount? Happy to send a minimal repro — same prompt called twice with caching on, asserting the cache attributes are absent and the total vs the Anthropic console's billed tokens diverge — or a small PR adding the cache fields per the semconv rule.
Context: I maintain AgentMeasure, a measurement-conformance project running a public 30-projects/30-days check of agent-telemetry semantics; this is the finding worth asking about directly. Not claiming a bug without your read on the intended convention.
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.