galaxyproject / galaxyproject/loom

/cost meter under-reports Anthropic spend ~6.5x (prices cache-creation tokens as cache reads)

Open
#273 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
TypeScript
Stars
14
Forks
12
Avg merge
6d 5h
Merged PRs (30d)
17

Description

## What happened

In a single analysis session the `/cost` meter reported ~$8 of Claude spend. The real Anthropic bill for the same session was $53.98 -- a ~6.5x under-report.

## Root cause (diagnosed by tester, well-supported)

Every step re-sends the full conversation, relying on Anthropic prompt caching. The default cache TTL is 5 minutes. This workload has many waits longer than that (long-running tool jobs, external fetches, Galaxy job polling, model switches, thinking time), so the cache expires and Anthropic re-creates it (billed at the `cache_creation` rate) instead of reusing it (`cache_read` rate).

The meter assumes everything was a cheap cache read, so it under-bills.

## Evidence

- Per-token math is correct at the app's own rates -- not a pricing-table arithmetic error.
- Cost is dominated by ~12.2M tokens the meter counts as cheap `cache_read`. Re-pricing those at the `cache_creation` rate yields ~$52, matching the bill.
- Source check: no cache TTL is set anywhere -> falls back to Anthropic's 5-min default. Pricing table also has no >200K-token tier even though both Claude models run with 1M-token context.

Reported token breakdown:
```
claude-opus-4-8
input 18 - output 12,791 - cache read 970,846 - cache write 164,232 -> meter $1.83
claude-sonnet-4-6
input 75 - output 49,471 - cache read 11,089,275 - cache write 578,829 -> meter $6.24
```

## Expected vs actual

- Expected: cost reflects what Anthropic actually billed, using the API's `cache_creation_input_tokens` vs `cache_read_input_tokens` from the usage block.
- Actual: meter prices (apparently) all cached tokens at the read rate, producing ~$8 vs the real ~$54.

## Fixes

1. Price from the actual usage breakdown the API returns (input / `cache_creation` / `cache_read` / output), not an assumed all-reads model.
2. Add a >200K-token price tier (both models configured for 1M context).
3. Warn the tester as they approach the $100 cap.
4. Consider a longer prompt-cache TTL (Anthropic supports a 1-hour cache option) so the cached prefix survives long tool waits -- this reduces real cost, separately from making the meter honest.

## Sysinfo

app 0.3.1 - darwin/arm64 - electron 41.2.1 - node 24.14.1 - provider anthropic / claude-sonnet-4-6

(Related: #32 cost dashboard -- that's display; this is computation correctness. Companion to the `/cost`-is-billable bug #263.)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.