anomalyco / anomalyco/models.dev
Model family field meaning
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 6.9k
- Forks
- 1.7k
- Avg merge
- 3h 21m
- Merged PRs (30d)
- 873
Description
I recently have been slightly irritated by how model families are used on models.dev, take for example, "grok-beta" and "grok-vision" being used on one single model each, or many of the opencode zen models that curiously have their own families which are -free.
It is my impression that model families are meant to be something along the lines of the model series or family, e.g. Kimi models (K2 K2.5 K2.6, K2-thinking, etc.) are all Kimi models, but not every provider agrees with what family they belong to. For K2.5 and K2.6 these are the family ids given by providers:
kimi-k2.5:
kimi (20): abacus, aihubmix, alibaba-cn, alibaba-coding-plan, alibaba-coding-plan-cn, azure, azure-cognitive-services, cloudflare-ai-gateway, cloudflare-workers-ai, digitalocean, hpc-ai, jiekou, nano-gpt, novita-ai, nvidia, ollama-cloud, opencode, openrouter, tencent-coding-plan, vercel
(none) (no family) (5): firmware, kilo, poe, qiniu-ai, zenmux
kimi-k2.5 (4): llmgateway, moonshotai, moonshotai-cn, opencode-go
kimi-thinking (1): cortecs
kimi-k2.6:
kimi-k2.6 (6): llmgateway, moonshotai, moonshotai-cn, nano-gpt, opencode-go, vercel
kimi (6): alibaba-cn, cloudflare-ai-gateway, cloudflare-workers-ai, novita-ai, opencode, openrouter
(none) (no family) (2): kilo, zenmux
kimi-thinking (1): cortecs
In my opinion, these should all be kimi, and this extends to many models.
I have written a script to help find models that have no family, have families that dont agree with other providers, or families that have an anomalous number of uses, or even none at all, with the help of GPT-5.4 (planning) and MiMo V2.5 Pro.
I am very interested in trying my best to improve the family for models that either have none, models that have families that dont make sense, or at the very minimum aligning the families and adding definitions to models that do not have a family. However, before I just assume that I know best, because I probably dont, I would like to have a concrete understanding of what the family key is truly meant to be.
My current assumption is that family should represent the base architecture/brand (e.g., llama, gpt, kimi) rather than specific versions (e.g., kimi-k2.5). Is this the intended direction? If not, why are all gpt models gpt and then a variant (e.g. gpt-codex, gpt-pro)? I would just like some more clarification on this field before I make an attempt to cleanup the data.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Review the model-family examples and the proposed kimi and gpt conventions in the issue, along with the author's family-analysis script. Done means the intended meaning of family is clarified and a concrete, agreed direction exists for correcting inconsistent or missing model-family data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- machine-learning
- Domain
- data, machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100