hoangsonww / hoangsonww/Claude-Code-Agent-Monitor

[Feature]: Cost what-if repricing simulator (model remap, fast-mode, intro-rate expiry)

Open
#304 0 comments 0 reactions 1 assignee Claimed by @hoangsonww View on GitHub
bug documentation enhancement good first issue help wanted question
Dominant language
TypeScript
Stars
1k
Forks
234
Avg merge
2d 9h
Merged PRs (30d)
18

Description

### What problem does this solve?

The pricing engine is now genuinely good — per-model rules, cache tiers, 1h vs 5m cache writes, fast-mode premiums, time-limited intro rates, GPT short/long context bands, server-tool request surcharges. All of it answers one question: **what did this cost?**

The question I actually have when I look at the number is a different one: **what would it have cost if I'd done it differently?** Specifically:

- What would last month have cost if the work had run on Sonnet instead of Opus? On Haiku?
- The intro pricing on a model expires at the end of the month — what does my current usage pattern cost the day after?
- Fast mode is a premium rate. Across my real history, what did enabling it actually cost me?
- My cache-write ratio is high. What would a better caching pattern be worth in dollars?

Every input needed to answer these is already in `token_usage`, bucketed by exactly the dimensions that determine the rate (`model`, `speed`, `inference_geo`, `service_tier`, `context_size`). The data is there; there's just no way to re-run the pricing math under a hypothetical.

### Proposed solution

A **repricing simulator** — deterministic, offline, no new ingestion, no new tables.

**Server** — `POST /api/pricing/simulate` taking a scope (date range, project, session ids, provider) and a list of transformations, returning the actual cost, the simulated cost, and a per-model / per-dimension delta breakdown:

```jsonc
{
"scope": { "since": "2026-07-01", "project": "/Users/me/work/api" },
"transforms": [
{ "type": "remap_model", "from": "claude-opus-*", "to": "claude-sonnet-5" },
{ "type": "set_speed", "value": "standard" },
{ "type": "expire_intro_rates", "as_of": "2026-09-01" }
]
}
```

Implementation is a pure function over the existing rate lookup: re-bucket the `token_usage` rows under the transformed dimensions and re-run the same pricing code path. Critically it must **reuse** the production rate resolver rather than reimplementing it, or the simulator will drift from reality and quietly become fiction.

**Honesty constraint** — a model remap is a cost model, not a behavior model. A cheaper model may need more turns to do the same job, and the UI must say so plainly ("assumes identical token usage; a different model may take more or fewer turns") instead of presenting the delta as a prediction. That caveat is part of the feature, not a footnote.

**UI** — a "What if" panel on Analytics: pick a scope, stack transformations, see actual vs. simulated side by side with a per-model waterfall of where the difference comes from. Saved scenarios so "post-intro-expiry pricing" is one click, not a re-entry.

**CLI** — `ccam cost --simulate 'model:opus→sonnet' --since 30d`, which is the form I'd use most, since the answer usually feeds a decision about which model to configure next.

### Alternatives considered

- **Export the data and do it in a spreadsheet.** Possible today, and it's what I've done — but it means reimplementing cache tiers, intro-rate cutoffs and the fast-mode multipliers by hand, which is exactly where a spreadsheet gets it wrong.
- **Temporarily edit the pricing rules and read the normal reports.** Destructive, affects live reporting for everyone using the dashboard, and can't express "expire intro rates as of a future date".
- **Bake it into scheduled reports (#7).** A generated report is a fine *destination* for a saved scenario, but the core need is interactive — you want to try five variations in a minute.

### Area

Settings / Pricing

### How important is this to you?

Would significantly improve my workflow

### Additional context

This depends on the per-`message.id` usage reconciliation from #293/#294 being correct — a simulator on top of inflated token totals would produce confidently wrong answers. Worth adding a test that the simulator with an empty transform list returns exactly the figure `/api/pricing/cost` reports, as a standing guard against drift between the two code paths.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.