anthropics / anthropics/claude-agent-sdk-python

get_context_usage() makes uncached API calls that are invisible to the message stream

Offen
#1,159 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
8.1k
Forks
1.3k
PR-Merge-Kennzahlen
PR-Kennzahlen ausstehend

Beschreibung

ClaudeSDKClient.get_context_usage() appears to count tokens by asking the model rather than locally. Each invocation fires a non-streaming request (InvokeModel) per registered item, with max_tokens: 1 and a filler user message:

json
{
"max_tokens": 1,
"messages": [{"role": "user", "content": "count"}],
"tools": [{ "name": "...", "description": "...", "input_schema": {...} }]
}

Two issues:

No caching. The requests carry no cache_control by default, so the same static content is billed as fresh input on every invocation. The measured items don't change over a session's lifetime, so caching the per-item counts would avoid re-measuring entirely.

Invisible to usage tracking. These calls don't appear on the SDK message stream and carry no usage block, so anything tracking spend from the stream misses them completely. We only found them via provider-side invocation logs. The method reads like cheap local introspection, and there's nothing in the return value or the docs indicating it costs API calls.

Environment: claude-agent-sdk 0.2.101, Bedrock

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.