anomalyco / anomalyco/models.dev

OpenRouter sync records promotional prices as standard prices

Open
#4,580 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
6.9k
Forks
1.7k
Avg merge
3h 21m
Merged PRs (30d)
873

Description

The catalog records promotional prices as if they were standard ones

GET /api/v1/models — the feed the OpenRouter sync reads — quotes a headline price that is already discounted when the route it came from is running a promotion. The payload has no field saying so, so the discount is invisible to the sync and the promo rate lands in the catalog as the model's price.

providers/openrouter/models/upstage/solar-pro4.toml is committed today as:

[cost]
input = 0.03
output = 0.12

That is a 90%-off promotion. The standard rate is $0.30 / $1.20. Nothing in the file records that, and when the promo ends the price will appear to jump 10x with no explanation in the diff.

This is provable from single-endpoint models, where there is no ambiguity about which route the headline price came from:

Model Endpoints discount In catalog Standard rate
upstage/solar-pro4 1 0.9 $0.03/M $0.30/M
inclusionai/ling-2.6-flash 1 0.9 $0.01/M $0.10/M
inclusionai/ring-2.6-1t 1 0.75 $0.075/M $0.30/M
meituan/longcat-2.0 1 0.6 $0.30/M $0.75/M
tencent/hy3-preview 1 0.65 $0.063/M $0.18/M

20 of the 350 OpenRouter models are currently affected.

Where the missing data lives

The per-model /api/v1/models/{author}/{slug}/endpoints route reports a discount fraction on each provider endpoint's pricing. OpenRouter also shows the pre-discount price struck through on the model's provider table — e.g. on deepseek-v4-flash-0731, StreamLake shows "35% off", $0.14 struck through, $0.091 charged.

The aggregate feed exposes neither.

Which discount belongs to the recorded price

The headline price does not say which endpoint it came from, which is the part that needs a decision. The rule that works: the endpoint quoting exactly the headline (prompt, completion) is the route being recorded, so its discount is the one consistent with the cost.input/cost.output already being written.

This matters because a model can have discounted endpoints that are not the one in the catalog. deepseek/deepseek-v4-flash-0731 has three discounted providers, but its headline $0.08/$0.252 is DigitalOcean's undiscounted row — so the correct answer for that model is no discount. Across all 350 models the rule produced zero ambiguous matches: 295 matched an undiscounted endpoint, 20 matched a discounted one, 10 matched none (all :free variants priced at 0).

Cost is one extra request per model; a full sweep is ~13s at a concurrency of 6.

Proposed shape

Keep prices meaning what they mean today — what you are billed — and add fields describing what has already been taken off them:

[cost]
input = 0.091      # billed
output = 0.182
discount = 0.35    # 35% off, already reflected above
input_list = 0.14  # standard rate
output_list = 0.28

*_list is price / (1 - discount), which is exactly the struck-through figure OpenRouter displays. It is derivable rather than new information, so it is a judgement call whether to store it — the argument for is that it makes the diff self-describing: when a promo ends, prices return to precisely the previously recorded input_list, distinguishing a promotion from a real price cut without recomputation.

Questions before this is worth merging

  1. Appetite. Is a promotional-price field in scope for what you want this dataset to be, or would you rather the catalog track standard rates only — in which case the fix is the inverse (record input_list as input) and this is still a bug worth closing.
  2. Store *_list, or just discount? The latter is smaller and loses nothing retrievable.
  3. Churn. A discount flips whenever a promo starts or ends. classifyAutoMerge gates on created/deleted counts and reasoning metadata — updated is computed but never gates anything, so discount-only changes would auto-merge unbounded. If that is unwanted it needs an update-side gate that does not exist today. (Measured over ~25 minutes none of the 20 moved, so the real rate may be low.)
  4. Scope. Wiring this for OpenRouter only leaves an authored discount on any other synced provider to be stripped on its next run, since those modules rebuild cost from their own APIs. Making it durable catalog-wide would need a runner-level preserve rule.

Implementation

#4576 implements the above for OpenRouter only, per sync.md's preference for small provider-specific PRs. Fail-soft (a model whose endpoint cannot be matched keeps its authored discount rather than churning), bun validate passes, 10 new tests, no regressions.

Happy to cut it down to just discount, rename the fields, or close it if this is out of scope — the decision on Q1 and Q2 changes what the PR should look like.

cc @rekram1-node @fwang

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the OpenRouter endpoint route described in the issue, sync.md, and implementation reference #4576. Review the 10 new tests and the classifyAutoMerge discussion, then resolve the requested decisions about promotional versus standard prices, stored fields, churn, and provider scope; done means the chosen behavior is covered without regressions.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
data
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.