AgentOps-AI / AgentOps-AI/agentops

Anthropic prompt caching: cache tokens not emitted on the Anthropic paths — is session total_tokens systematically low?

未关闭
#1,445 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
5.8k
派生
619
PR 合并指标
30 天内没有已合并 PR

描述

A semantics question found while auditing token accounting across agent-telemetry SDKs.

**The observation**

On the Anthropic paths, usage extraction reads only `usage.input_tokens` / `usage.output_tokens` and computes `total = input + output`:

- non-streaming: `agentops/instrumentation/providers/anthropic/attributes/message.py` (~L319-332)
- streaming: `.../anthropic/stream_wrapper.py` (~L167-177 and ~L406-418)

Anthropic's `cache_read_input_tokens` / `cache_creation_input_tokens` are separate fields (not included in `input_tokens`), and I don't find them read anywhere on these paths — `LLM_USAGE_CACHE_READ_INPUT_TOKENS` / `CACHE_CREATION_INPUT_TOKENS` are defined in `agentops/semconv/span_attributes.py` but have no setter calls in the Anthropic instrumentation.

Meanwhile the OpenAI Responses path (`providers/openai/attributes/response.py`, ~L296-307) does emit `cached_tokens → cache_read` and `reasoning_tokens` — so the asymmetry is within one SDK.

**Why it matters**

The OTel GenAI semantic conventions for Anthropic (`semantic-conventions-genai`, `docs/gen-ai/anthropic.md`) are explicit: cache read/write tokens **MUST be added** to the native `input_tokens` to compute `gen_ai.usage.input_tokens`. If they're dropped instead, a prompt-caching Claude session reports `total_tokens` systematically below what the provider actually meters (cache reads are billed, just at a lower rate — they're not free tokens).

**The question**

Is this asymmetry intentional (e.g. a deliberate "native usage only" convention on the Anthropic path)? If not: for sessions with `cache_control` enabled, does any downstream token/cost figure derived from these spans undercount? Happy to send a minimal repro — same prompt called twice with caching on, asserting the cache attributes are absent and the total vs the Anthropic console's billed tokens diverge — or a small PR adding the cache fields per the semconv rule.

Context: I maintain AgentMeasure, a measurement-conformance project running a public 30-projects/30-days check of agent-telemetry semantics; this is the finding worth asking about directly. Not claiming a bug without your read on the intended convention.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。