GLM-5.3-Flash: which Transformers version produced the published HF logit parity?

The GLM-5.3-Flash model-coverage page reports a Hugging Face logit parity result and, in the same page, states which Transformers version the training path needs. I could not reproduce the parity from the environment the page prescribes, because the Transformers-native implementation of this model does not exist at the pinned version.

Filing this as a question rather than a bug — the most likely explanation is that the parity was produced with a newer or pre-release Transformers than the training path pins, in which case the page may just need a note.

What the page says

Supported Training Path, L44-L45:

The implementation runs on the repository's pinned transformers==5.12.1; users do not need Transformers 5.16, a nightly Transformers build, or trust_remote_code.

Numerical Validation → Hugging Face Logit Parity, L122-L133:

A four-layer model containing KDA, KDA, KDA, and DSA was compared with the Transformers reference at sequence length 4,096 over the full 154,880-token vocabulary. All 135 expected tensors loaded with no missing, unexpected, or shape-mismatched keys.

Metric	Result
Mean KL, HF to NeMo AutoModel	0.0002477953
P95 KL	0.0005324535
Max KL	0.0271428488
Logits cosine similarity	0.9999251899
Top-1 token agreement	98.33984375%

What I measured

transformers.models.glm5_next is absent from 5.12.1, 5.15.1 and 5.16.0, and first appears in 5.17.0. Checked by listing transformers/models/* inside each released wheel:

• 5.15.1 — glm, glm4, glm46v, glm4_moe, glm4_moe_lite, glm4v, glm4v_moe, glm_image, glm_moe_dsa, glm_ocr, glmasr, glmga, xglm — no glm5_next
• 5.16.0 — same set, no glm5_next
• 5.17.0 — adds configuration_glm5_next.py, modeling_glm5_next.py, modular_glm5_next.py, processing_glm5_next.py, and the image/video processors

The zai-org/GLM-5.3-Flash repository contains no .py files, so the trust_remote_code path is not available either.

On nvcr.io/nvidia/nemo-automodel @sha256:c21c0c3b82f68fcccfded4bf9fe11b2d53df1c9a969e9e883a1562ffca6294f6, transformers.__version__ is 5.12.1 and glm5_next is likewise absent.

Why this matters (and what is unaffected)

Unaffected: running and training the model. AutoModel ships its own glm5_next implementation, so it does not need Transformers to carry the model. I loaded a four-layer language-only slice successfully under the image above with a 9a463234 source overlay — 110 expected keys, zero missing, zero unexpected, FP8 dequantization and grouped-expert aggregation both clean.

Affected: the reference. A fixed-input logit comparison needs a second independent implementation, and in the prescribed environment there is not one available, so a reader following the page cannot reproduce the published parity numbers.

For contrast, the sibling Qwen3.8-Flash-Next page compares against an SGLang reference and makes no Transformers version claim, so this looks specific to the GLM page rather than systemic.

Follow-up: the two implementations are not structurally identical

After filing, I built the reference side in a separate virtualenv (transformers==5.17.0, same torch and container) and constructed the same four-layer slice on both sides. They differ in two places:

	AutoModel glm5_next	Transformers 5.17.0 glm5_next
Four-layer expected keys	457 (110 language + 347 vision)	451 (104 language + 347 vision)
Grouped experts	experts.gate_and_up_projs / experts.down_projs	experts.gate_up_proj / experts.down_proj
mHC	attn_hc.fn, attn_hc._fp32_params.base, attn_hc._fp32_params.scale	attn_hc.fn, attn_hc.base, attn_hc.scale
KDA short conv	self_attn.q_conv1d, k_conv1d, v_conv1d (three)	self_attn.conv1d (one, fused)
KDA decay	self_attn._fp32_params.A_log	self_attn.forget_gate.A_log

The 110 vs 104 difference is exactly the short-conv decomposition: three KDA layers times two extra conv tensors each. The two should be numerically equivalent, but this is the kind of structural divergence that bears on question 3 below.

Also worth noting for anyone attempting a reproduction: only 63 of the 451 Transformers-side keys match the released checkpoint's names exactly, so the Transformers side performs its own renaming and expert grouping on load as well. A reproduction therefore has to go through from_pretrained on both sides rather than a manual load_state_dict, or the comparison stops being between two independent implementations.

Questions

1. Which Transformers version (or pre-release build) produced the published HF logit parity?
2. Given the training path pins 5.15.1, would it make sense for the page to point readers at a separate reference environment for reproducing the parity?
3. Does "All 135 expected tensors loaded" count AutoModel-side expected keys? A four-layer language-only stub built from the released config gives 110 on the AutoModel side and 104 on the Transformers side, so I may be constructing a different stub than the one measured. If the intended stub differs, I would be glad to know how.

Environment

AutoModel source   9a4632347323fbfee9ba1ae1a323669ef74fc5fb (overlay on the image below)
image              nvcr.io/nvidia/nemo-automodel@sha256:c21c0c3b82f68fcccfded4bf9fe11b2d53df1c9a969e9e883a1562ffca6294f6
torch              2.13.0a0+8145d630e8.nv26.06, CUDA 13.3
GPU                NVIDIA B200, compute capability 10.0
model revision     zai-org/GLM-5.3-Flash @ eb9eb208eb0d988989d07a6a12d0fdeb5f52574a
backend            attn=sdpa, experts=torch, dispatcher=torch, rms_norm=torch_fp32
reference venv     transformers==5.17.0, same torch and container

Related: #3939 fixes two smaller factual issues on the same page.