anthropics / anthropics/claude-agent-sdk-python

get_context_usage() makes uncached API calls that are invisible to the message stream

Ouverte
#1,159 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
8.1k
Forks
1.3k
Métriques de merge des PR
Métriques de PR en attente

Description

ClaudeSDKClient.get_context_usage() appears to count tokens by asking the model rather than locally. Each invocation fires a non-streaming request (InvokeModel) per registered item, with max_tokens: 1 and a filler user message:

json
{
"max_tokens": 1,
"messages": [{"role": "user", "content": "count"}],
"tools": [{ "name": "...", "description": "...", "input_schema": {...} }]
}

Two issues:

No caching. The requests carry no cache_control by default, so the same static content is billed as fresh input on every invocation. The measured items don't change over a session's lifetime, so caching the per-item counts would avoid re-measuring entirely.

Invisible to usage tracking. These calls don't appear on the SDK message stream and carry no usage block, so anything tracking spend from the stream misses them completely. We only found them via provider-side invocation logs. The method reads like cheap local introspection, and there's nothing in the return value or the docs indicating it costs API calls.

Environment: claude-agent-sdk 0.2.101, Bedrock

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.