stacklok / stacklok/mecatl

Cost-aware model usage and delegation visibility

Open
#610 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Go
Stars
152
Forks
16
Avg merge
14h 48m
Merged PRs (30d)
536

Description

Problem

Mecatl records provider-neutral token usage and supports token ceilings, but it cannot reliably answer:

  • What did a session or delegation tree cost?
  • Which harness functions and models drove that spend?
  • How much of the estimate is based on known pricing?
  • Did caching materially reduce cost?
  • Which currently available models are economical alternatives?
  • Is an external budget authority warning that a configured threshold is approaching?

The embedded models.dev corpus already retains price data, but the catalog parser intentionally ignores it. Live model listing identifies what a credential can use but normally does not provide pricing. Some LLM calls also sit outside the main turn-level usage totals, preventing complete attribution.

MVP outcome

Give operators useful cost visibility and enough awareness to make modest optimizations without turning mecatl into a billing or autonomous financial-control system.

The MVP should answer:

  1. What has this session probably cost?
  2. What functions and models consumed that cost?
  3. How much usage could be priced, and from which source?
  4. How much did prompt caching probably save?
  5. Which economical models are currently available?
  6. Has an external budget authority reported an approaching threshold?

Product principles

  • Tokens and money remain separate controls.
  • Unknown price is never treated as zero or free.
  • Every amount is explicitly estimated unless reconciled by an authoritative source.
  • Price provenance and price-card freshness are visible.
  • Pricing uses the actually served model when known; requested/resolved/fallback identities remain inspectable.
  • Cost metadata never enters prompt text or invalidates prompt caches.
  • Provider-private billing knobs do not widen port.LLMRequest.
  • Default behavior remains unchanged when pricing or advisory data is absent.
  • External budget advisories and local session estimates remain separate signals; mecatl does not derive a shared remaining balance from them.

MVP scope

  • Parse and expose models.dev list pricing, including cache and tiered/context-sensitive prices.
  • Join public catalog metadata with the existing resolved live model inventory.
  • Expose a bounded, filterable AvailableModels tool over that same inventory.
  • Record one function-attributed observation for every LLM invocation.
  • Attribute requested, resolved, served, retried, and fallback model usage where available.
  • Calculate versioned list-price estimates with explicit coverage and provenance.
  • Provide a machine-readable session cost summary and a basic TUI view/live meter.
  • Break down estimated spend by function, model, successful/failed/retry usage, and cache economics.
  • Consume and surface provider-neutral external budget advisories without exposing deployment-specific transport details.

Explicitly deferred

  • Hard monetary enforcement and concurrent reservation/settlement.
  • Automatic cost-aware routing.
  • Contract-price administration and invoice reconciliation.
  • Organization-wide chargeback or historical analytics.
  • LLM-generated semantic cost labels.
  • Detailed diagnosis of why a prompt-cache miss occurred.
  • Cost forecasting and cost-per-PR/outcome optimization.

Success criteria

  • A completed session has an obtainable estimated-cost report with pricing coverage and unpriced usage.
  • The report reconciles all observed LLM calls without double-counting nested delegation.
  • Main, compaction, routing, safety, learning, Subagent, Parallel, and Team costs can be distinguished.
  • Requested versus actually served/fallback model attribution is honest when route metadata exists and explicitly unknown otherwise.
  • Cache-read/write cost, estimated uncached baseline, and estimated savings are visible.
  • The model inventory exposes economical available alternatives without claiming equivalent quality.
  • An external advisory can produce a bounded, deduplicated client-visible warning while ordinary deployments remain byte-identical.
  • Tests remain offline and no provider or gateway is required.

Child outcomes

This epic is delivered by the attached MVP sub-issues:

  1. Model pricing, enriched inventory, and AvailableModels.
  2. Complete function/model-attributed LLM usage and estimates.
  3. Cost report, live TUI summary, and basic cache economics.
  4. Provider-neutral external budget-advisory consumption.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or entry points; start by reviewing the MVP sub-issues for model pricing, attributed usage, cost reporting, and budget advisories. Done means the scoped MVP provides an estimated session-cost report, reconciles observed calls without double-counting, exposes attribution and cache economics, and surfaces bounded external advisories with offline tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
ai-infra-agents, observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.