anomalyco / anomalyco/models.dev

Proposal: objective sortable ranking columns (Overall / Value / Capability)

Open
#1,893 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
6.9k
Forks
1.7k
Avg merge
3h 21m
Merged PRs (30d)
873

Description

Proposal: objective, sortable ranking columns

models.dev already exposes every fact needed to rank models objectively (cost, context window, output limit, capability flags, modality breadth, release date) — but there's no way to rank the catalog as a whole; you can only sort one column at a time.

I'd like to add three transparently-computed, sortable score columns — Overall, Value, Capability — derived purely from those existing fields (no benchmarks, no hand-grading), plus a dynamic rank (#) column.

Why columns, not one ranking

Any single "best model" score is an opinion. Shipping it as sortable columns keeps the data neutral and lets each user pick the lens that fits their use case (cheap-yet-capable vs. raw feature breadth vs. balanced).

Honest limitation

The catalog has no quality/benchmark field, so this measures spec-breadth-per-dollar, not model intelligence. Broad/cheap/omni-modal models (and auto-routers) rank highest — correct given the inputs, but worth stating plainly. If an objective quality signal ever lands in the schema, it drops straight into the existing blend.

Implementation

Web-only; canonical api.json is unchanged. Weights live in one documented place so they're easy to audit or tune.

PR: #1892

Open to trimming the column count, making weights configurable, or holding this if a built-in ranking isn't a direction you want. Feedback welcome.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing PR #1892, which is linked from the proposal, to understand the current implementation and its open decisions. Completion means the web catalog provides auditable Overall, Value, and Capability sortable columns plus a dynamic rank column, while leaving the canonical api.json unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.