feat(schema): track LLM cache-creation (write) tokens in PromptTokenDetails
- Dominant language
- Go
- Stars
- 13k
- Forks
- 1.1k
- Avg merge
- 4h 6m
- Merged PRs (30d)
- 41
Description
**Is your feature request related to a problem? Please describe.**
`schema.PromptTokenDetails` models only CachedTokens (cache read). There's no field for cache write (creation) tokens. Multiple providers bill cache writes at a premium and report a distinct count - Anthropic cache_creation_input_tokens (1.25x/2x), OpenAI cache_write_tokens (newer manual-cache models, 1.25x), AWS Bedrock cacheWriteInputTokens - but eino has nowhere to put it, so bindings fold it into `PromptTokens`. The total is preserved, but the distinct write count isn't surfaced:
```go
promptTokens := int(usage.InputTokens + usage.CacheReadInputTokens + usage.CacheCreationInputTokens)
PromptTokenDetails{ CachedTokens: int(usage.CacheReadInputTokens) } // write count not surfaced
```
Result: write tokens are counted, but unattributable - they land in the input bucket and get priced at 1x instead of the provider's write rate, so cost tracking under-reports the write premium and can't break out write volume.
**Describe the solution you'd like**
```go
type PromptTokenDetails struct {
CachedTokens int `json:"cached_tokens"` // cache read
CacheCreationTokens int `json:"cache_creation_tokens"` // cache write
}
```
**Describe alternatives you've considered**
- Compute downstream - impossible; the write count is already summed into `PromptTokens` at the binding and can't be separated back out.
- Carry via ResponseMeta.Extra - works but unshaped and per-consumer; a typed field is the right home.
**Additional context**
Every other provider-agnostic framework already models write separately and provider-neutrally: LangChain / LangChain.js (input_token_details.cache_creation), Vercel AI SDK (inputTokens.cacheWrite), LlamaIndex (cache_creation_input_tokens). Gemini is the one provider with no per-request write count (write billed in a separate caches.create call) — a neutral field simply stays zero there.
Contributor guide
Research direction
Start at the Go definition of schema.PromptTokenDetails and trace the provider bindings that construct it from cache usage. Add the distinct cache-creation token field described in the issue, preserving existing cached-token behavior, and verify that provider-reported write counts are surfaced without changing total prompt-token accounting.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- ai
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 65/100