Support cached-input pricing for custom providers
- 主要語言
- Rust
- 星號
- 54.2k
- 分支
- 6.2k
- 平均合併
- 3 天 2 小時
- 30 天內合併 PR
- 262
描述
## Problem
Custom provider model metadata currently supports `input_token_cost` and `output_token_cost`, but it cannot specify a separate price for cached input tokens.
Some OpenAI-compatible providers return cached-token usage and bill those tokens at a substantially discounted rate. For example, Cloudflare Workers AI enables prefix caching for supported models, reports cached token counts in the response `usage` object, and applies a different cached-input price.
Because custom providers cannot configure that price separately, Goose's estimated and accumulated session cost can materially overestimate actual provider billing during long agent sessions with high cache-hit rates.
## Who this affects
Users of custom OpenAI-compatible providers that:
- report cached input tokens;
- charge a distinct cached-input rate;
- use long, repeated prompt prefixes, particularly coding-agent sessions.
Cloudflare Workers AI is one example, but the capability would be useful for any compatible provider with discounted cached input.
## Current behavior
A custom model can define:
- `input_token_cost`
- `output_token_cost`
There is no model metadata field for cached-input cost. Goose already has usage fields for cache-read and cache-write tokens, but custom-provider pricing cannot use a separate cached-input rate.
## Expected behavior
Custom providers should be able to optionally specify a cached-input token price.
When both cached-token usage and cached pricing are available, Goose should use them when calculating session cost. Providers that do not configure cached pricing should retain the current behavior.
The capability should work consistently in session totals and any CLI or Desktop cost display.
## Verification
A suitable verification plan would cover:
- no cached tokens;
- partially cached input;
- fully cached input;
- cached-token usage without cached pricing;
- cached pricing without cached-token usage;
- backward compatibility with existing custom-provider JSON files;
- accumulated cost across multiple requests.
## Example provider documentation
Cloudflare Workers AI prompt caching:
https://developers.cloudflare.com/workers-ai/features/prompt-caching/
Cloudflare Workers AI pricing:
https://developers.cloudflare.com/workers-ai/platform/pricing/
貢獻指南
評估
這個 Issue 還沒有評估資料。