anomalyco / anomalyco/models.dev

Model family field meaning

Open
#1,599 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
6.9k
Forks
1.7k
Avg merge
3h 21m
Merged PRs (30d)
873

Description

I recently have been slightly irritated by how model families are used on models.dev, take for example, "grok-beta" and "grok-vision" being used on one single model each, or many of the opencode zen models that curiously have their own families which are -free.

It is my impression that model families are meant to be something along the lines of the model series or family, e.g. Kimi models (K2 K2.5 K2.6, K2-thinking, etc.) are all Kimi models, but not every provider agrees with what family they belong to. For K2.5 and K2.6 these are the family ids given by providers:

  kimi-k2.5:
    kimi (20): abacus, aihubmix, alibaba-cn, alibaba-coding-plan, alibaba-coding-plan-cn, azure, azure-cognitive-services, cloudflare-ai-gateway, cloudflare-workers-ai, digitalocean, hpc-ai, jiekou, nano-gpt, novita-ai, nvidia, ollama-cloud, opencode, openrouter, tencent-coding-plan, vercel
    (none) (no family) (5): firmware, kilo, poe, qiniu-ai, zenmux
    kimi-k2.5 (4): llmgateway, moonshotai, moonshotai-cn, opencode-go
    kimi-thinking (1): cortecs

  kimi-k2.6:
    kimi-k2.6 (6): llmgateway, moonshotai, moonshotai-cn, nano-gpt, opencode-go, vercel
    kimi (6): alibaba-cn, cloudflare-ai-gateway, cloudflare-workers-ai, novita-ai, opencode, openrouter
    (none) (no family) (2): kilo, zenmux
    kimi-thinking (1): cortecs

In my opinion, these should all be kimi, and this extends to many models.

I have written a script to help find models that have no family, have families that dont agree with other providers, or families that have an anomalous number of uses, or even none at all, with the help of GPT-5.4 (planning) and MiMo V2.5 Pro.

I am very interested in trying my best to improve the family for models that either have none, models that have families that dont make sense, or at the very minimum aligning the families and adding definitions to models that do not have a family. However, before I just assume that I know best, because I probably dont, I would like to have a concrete understanding of what the family key is truly meant to be.

My current assumption is that family should represent the base architecture/brand (e.g., llama, gpt, kimi) rather than specific versions (e.g., kimi-k2.5). Is this the intended direction? If not, why are all gpt models gpt and then a variant (e.g. gpt-codex, gpt-pro)? I would just like some more clarification on this field before I make an attempt to cleanup the data.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the model-family examples and the proposed kimi and gpt conventions in the issue, along with the author's family-analysis script. Done means the intended meaning of family is clarified and a concrete, agreed direction exists for correcting inconsistent or missing model-family data.

Written by the indexing model from the issue text.

Assessment

Tech stack
machine-learning
Domain
data, machine-learning
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.