Cost-aware model usage and delegation visibility
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 152
- Forks
- 16
- Avg merge
- 14h 48m
- Merged PRs (30d)
- 536
Description
Problem
Mecatl records provider-neutral token usage and supports token ceilings, but it cannot reliably answer:
- What did a session or delegation tree cost?
- Which harness functions and models drove that spend?
- How much of the estimate is based on known pricing?
- Did caching materially reduce cost?
- Which currently available models are economical alternatives?
- Is an external budget authority warning that a configured threshold is approaching?
The embedded models.dev corpus already retains price data, but the catalog parser intentionally ignores it. Live model listing identifies what a credential can use but normally does not provide pricing. Some LLM calls also sit outside the main turn-level usage totals, preventing complete attribution.
MVP outcome
Give operators useful cost visibility and enough awareness to make modest optimizations without turning mecatl into a billing or autonomous financial-control system.
The MVP should answer:
- What has this session probably cost?
- What functions and models consumed that cost?
- How much usage could be priced, and from which source?
- How much did prompt caching probably save?
- Which economical models are currently available?
- Has an external budget authority reported an approaching threshold?
Product principles
- Tokens and money remain separate controls.
- Unknown price is never treated as zero or free.
- Every amount is explicitly estimated unless reconciled by an authoritative source.
- Price provenance and price-card freshness are visible.
- Pricing uses the actually served model when known; requested/resolved/fallback identities remain inspectable.
- Cost metadata never enters prompt text or invalidates prompt caches.
- Provider-private billing knobs do not widen
port.LLMRequest. - Default behavior remains unchanged when pricing or advisory data is absent.
- External budget advisories and local session estimates remain separate signals; mecatl does not derive a shared remaining balance from them.
MVP scope
- Parse and expose models.dev list pricing, including cache and tiered/context-sensitive prices.
- Join public catalog metadata with the existing resolved live model inventory.
- Expose a bounded, filterable
AvailableModelstool over that same inventory. - Record one function-attributed observation for every LLM invocation.
- Attribute requested, resolved, served, retried, and fallback model usage where available.
- Calculate versioned list-price estimates with explicit coverage and provenance.
- Provide a machine-readable session cost summary and a basic TUI view/live meter.
- Break down estimated spend by function, model, successful/failed/retry usage, and cache economics.
- Consume and surface provider-neutral external budget advisories without exposing deployment-specific transport details.
Explicitly deferred
- Hard monetary enforcement and concurrent reservation/settlement.
- Automatic cost-aware routing.
- Contract-price administration and invoice reconciliation.
- Organization-wide chargeback or historical analytics.
- LLM-generated semantic cost labels.
- Detailed diagnosis of why a prompt-cache miss occurred.
- Cost forecasting and cost-per-PR/outcome optimization.
Success criteria
- A completed session has an obtainable estimated-cost report with pricing coverage and unpriced usage.
- The report reconciles all observed LLM calls without double-counting nested delegation.
- Main, compaction, routing, safety, learning, Subagent, Parallel, and Team costs can be distinguished.
- Requested versus actually served/fallback model attribution is honest when route metadata exists and explicitly unknown otherwise.
- Cache-read/write cost, estimated uncached baseline, and estimated savings are visible.
- The model inventory exposes economical available alternatives without claiming equivalent quality.
- An external advisory can produce a bounded, deduplicated client-visible warning while ordinary deployments remain byte-identical.
- Tests remain offline and no provider or gateway is required.
Child outcomes
This epic is delivered by the attached MVP sub-issues:
- Model pricing, enriched inventory, and
AvailableModels. - Complete function/model-attributed LLM usage and estimates.
- Cost report, live TUI summary, and basic cache economics.
- Provider-neutral external budget-advisory consumption.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points; start by reviewing the MVP sub-issues for model pricing, attributed usage, cost reporting, and budget advisories. Done means the scoped MVP provides an estimated session-cost report, reconciles observed calls without double-counting, exposes attribution and cache economics, and surfaces bounded external advisories with offline tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- ai-infra-agents, observability-sre
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100