stacklok / stacklok/mecatl

Add model pricing, enriched inventory, and AvailableModels

Open
#611 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Go
Stars
152
Forks
16
Avg merge
14h 48m
Merged PRs (30d)
536

Description

Parent

Part of #610.

User outcome

An operator or agent can inspect the models that are actually available to the configured credentials and compare their public list-price economics alongside capabilities and context limits.

Current state

  • internal/adapter/providercatalog/models.dev.curated.json already retains model cost data.
  • internal/adapter/providercatalog/catalog.go intentionally ignores that data today.
  • ListModels already joins available providers with static/live model metadata and supports on-demand refresh plus last-known-good behavior.
  • The OpenAI-compatible /v1/models protocol normally provides availability and basic metadata, not pricing.

Scope

Price model

Parse the complete models.dev pricing shape required by the curated corpus, including where present:

  • ordinary input;
  • output;
  • cache read;
  • cache write;
  • tiered or context-sensitive rates;
  • units/currency;
  • source and last_updated provenance.

Requirements:

  • represent money without floating-point calculation errors;
  • preserve unknown price distinctly from zero;
  • preserve the price-card/source version used for an estimate;
  • call models.dev values public list-price estimates, never actual billing;
  • keep room for later operator/gateway overrides without implementing them here.
Enriched resolved inventory

Join availability and metadata using one inventory path:

live provider listing (availability)
+ embedded models.dev metadata and list price
= resolved inventory

A live-only model with no catalog match remains available with unknown price. A catalog model not available to the credential must not be resurrected by enrichment.

Extend ListModels additively with bounded, secret-free optional economics/provenance fields.

AvailableModels tool

Add a read-only, bounded tool over the same resolved inventory—not a second discovery subsystem. It should support a small useful filter set, for example:

  • provider;
  • required capabilities;
  • minimum context;
  • maximum known input/output price;
  • sort by price;
  • result limit.

Results must distinguish unavailable, unknown-price, and known-price models and must not claim that a cheaper model has equivalent quality.

Acceptance criteria

  • Every pricing shape present in the curated models.dev fixture either round-trips into the neutral price model or is deliberately rejected by a pinning test.
  • Unknown price remains unknown through catalog, resolved inventory, wire projection, and tool output.
  • ListModels continues to expose only available providers/models and remains backward compatible when economics fields are absent.
  • A live-only private model is returned with unknown price; a catalog-only unavailable model is not returned.
  • Price source, freshness/version, units, and estimated/list-price status are available to consumers.
  • AvailableModels is read-only, result-bounded, filterable, and consumes the same inventory snapshot/refresher as ListModels.
  • Tool output is compact and model-visible; deleting its discoverability instruction fails a real factory-path prompt test.
  • Tests use fixtures only and make no live network calls.
  • The public model API and user documentation are updated additively.

Out of scope

  • Automatic model selection or routing.
  • Contract pricing and invoice reconciliation.
  • Hard budgets.
  • Quality equivalence or benchmark scoring.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing ListModels and internal/adapter/providercatalog/catalog.go, then inspect the pricing data in internal/adapter/providercatalog/models.dev.curated.json. Map the shared resolved inventory before adding AvailableModels, and review the public model API and user documentation requirements. Done means pricing shapes, unknown values, availability, provenance, bounded filters, and backward compatibility are covered by fixture-based tests without live network calls.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.