repowise-dev / repowise-dev/repowise

[Bug] The Anthropic API provider counts only the uncached prompt remainder as input_tokens

Open
#2,367 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug help wanted
Dominant language
Python
Stars
6.7k
Forks
711
Avg merge
1d 13h
Merged PRs (30d)
439

Description

Summary

claude_cli counted only the uncached remainder of a prompt as input_tokens, which #2267 reported and #2337 fixed. The direct Anthropic API provider has the same undercount, and it has not been fixed.

packages/core/src/repowise/core/providers/llm/anthropic.py:234

input_tokens=response.usage.input_tokens,

The two cache figures are collected four lines later and kept only inside the usage dict:

:242-246

"cache_creation_input_tokens": getattr(
    response.usage, "cache_creation_input_tokens", 0
),
"cache_read_input_tokens": cached,

So the persisted input_tokens is the uncached remainder alone, and the two cached dispositions never reach the total.

Why this one matters more than #2267

claude_cli is billed against a subscription seat, so its ledger rows price at $0.00 and the undercount distorted token volume only. This provider is billed per token against an API key. Prompt caching is on for wiki page generation, where the prompt is large and stable and therefore mostly cache reads after the first call, which is exactly the shape that makes the missing portion the majority of the prompt.

The consequence is that repowise costs and the cost ledger under-report both input tokens and the dollar figure derived from them, on the provider where the dollar figure is real.

Not yet sized

I have not measured how far off the totals are on a real run, and the ratio depends on cache hit rate, so treat the mechanism as certain and the magnitude as unknown.

Done looks like

input_tokens carries the whole prompt, matching what #2337 established for claude_cli: the uncached remainder plus cache reads plus cache creation, with the uncached figure kept separately so nothing is lost.

Two things worth doing in the same pass. Check :251 and :267, which pass result.input_tokens onward, so they inherit whatever the fix decides. And pin the result with a recorded usage payload in a test, since the disjointness of the three fields is a property of the API contract and not something the repository can prove on its own.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in packages/core/src/repowise/core/providers/llm/anthropic.py at lines 234 and 242-246, then inspect the callers at lines 251 and 267 and the analogous fix in #2337. Record a usage payload covering uncached, cache-read, and cache-creation counts, verify the persisted total and separate usage fields, and run the relevant provider tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.