anomalyco / anomalyco/models.dev

Inconsistent radix for limit.context on Zhipu GLM models (200K stored as both 200000 and 204800)

Open
#3,738 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
6.9k
Forks
1.7k
Avg merge
3h 21m
Merged PRs (30d)
873

Description

Problem

The limit.context field for Zhipu GLM models is stored with an inconsistent radix across entries that represent the same documented context window.

Zhipu's official docs express context only as 1M / 200K / 128K (no precise integers), except glm-5.2's 1M which is decimal 1,000,000. The registry currently stores the same documented value two different ways:

Documented Stored as (decimal) Stored as (binary, ×1024)
200K 200000 (glm-5.1, glm-4.7-flash, glm-4.7-flashx, glm-5v-turbo) 204800 (glm-5, glm-4.6, glm-4.7)
128K 128000 (glm-4.6v) 131072 (glm-4.5 series)

Meanwhile limit.output is consistently binary (131072 / 98304 / 32768 / 16384), confirmed against Zhipu's official max_tokens table: https://docs.z.ai/guides/overview/concept-param

So context and output use different conventions, and context is internally inconsistent.

Impact

Any field derived from context inherits the ambiguity. In particular, limit.input = context - output (the convention used by the registry and by PR #3739 adding limit.input for GLM models) produces values that differ by 4800 (for the 200K group) or 3072 (for the 128K group) depending on which radix is "correct". Downstream tools computing thresholds against these limits get slightly different behavior per model with no clear ground truth.

Ask

Could maintainers confirm, against Zhipu's official spec, whether 200K / 128K context windows should be decimal (200000 / 128000) or binary (204800 / 131072), and normalize the zhipuai (and zai) entries accordingly? Once normalized, the limit.input values from #3739 can be recomputed if needed.

Also noticed

zhipuai/models/glm-4.6v-flash.toml is a broken symlink — it points to ../../zai/models/glm-4.6v-flash.toml, which does not exist in the repo. Pre-existing, unrelated to this issue, but flagging it here for cleanup.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the zhipuai and zai model registry entries, including the context and output limits described in the issue, and compare them with Zhipu's linked official max_tokens documentation. Confirm the intended radix with maintainers, then normalize the affected TOML entries and recompute limit.input where needed. Verify that corresponding model values are consistent; the broken glm-4.6v-flash symlink is separately noted and unrelated.

Written by the indexing model from the issue text.

Assessment

Domain
ai, data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.