mudler / mudler/vllm.cpp

The neartie gap artifacts are identically zero in 12 of 18 golden dirs while our_ids diverges: not measurements

Open
#2,929 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: - (owned under ## Owed in .agents/specs/glm4-moe-lite-router-f32.md)

neartie_gap_mnats.npy is the artifact every near-tie-banded model gate reads to
decide whether a divergence is admissible. In 12 of the 18 golden directories
that carry one it is identically zero at every position, while our_ids.npy
diverges from greedy_ids.npy at 13 to 83 positions in the same directory.
Measured at c796fea41:

dir                          gap min/max  nonzero   our_ids vs greedy_ids
deepseek_v2_greedy               0/250        1/128        36/128 divergent
glm4_moe_lite_greedy               0/0        0/128        59/128 divergent
internlm2_greedy_1_8b              0/0        0/256        42/256 divergent
internlm3_greedy_8b                0/0        0/256        22/256 divergent
llama_greedy_1b                    0/0        0/256        13/256 divergent
minicpm3_greedy_4b                 0/0        0/256        20/256 divergent
minicpm_greedy_2b                  0/0        0/256        34/256 divergent
mistral_greedy_7b                  0/0        0/256         6/256 divergent
olmo2_greedy_1b                   0/94        2/256        15/256 divergent
phi2_greedy_2_7b                 0/250        6/256        75/256 divergent
phi4_14b_greedy                    0/0        0/256        37/256 divergent
phi4_mini_greedy                0/1000        9/256        83/256 divergent
qwen35_greedy_0_8b               0/125        2/256         8/256 divergent
qwen3_32b_nvfp4a16_greedy          0/0         0/96        43/96  divergent
qwen3_greedy_0_6b                0/125        1/256        55/256 divergent
qwen3_greedy_4b                  0/250        2/256        43/256 divergent
qwen3coder_greedy                  0/0         0/96         1/96  divergent
yi_greedy_coder_1_5b             0/125        2/256        38/256 divergent

Reproduce with numpy over tests/parity/goldens/*/; no checkpoint and no GPU
are needed.

Why zero is not a possible measurement at a FIRST divergence

The generating scripts (scripts/*-neartie-gap.py) teacher-force the pinned
oracle on OUR token sequence and record
max(0, argmax_logprob - our_token_logprob) in milli-nats. At a prompt's first
divergent position our prefix is byte-identical to the oracle's, so the oracle's
teacher-forced argmax there is by construction the token its own free-running
greedy decode emitted — greedy_ids[i,j] — and ours is a different token. The
gap must therefore be strictly positive unless the two logprobs are exactly
equal in the oracle's own float output.

For glm4_moe_lite_greedy it reads 0 at all seven first divergences (prompts
0..6, positions 14, 6, 6, 4, 4, 4, 6), and the scripts' 99_999_000
outside-top-20 sentinel appears nowhere in the file. our_ids.npy is written by
the same script in the same call as the gap array, so the script demonstrably
ran to completion; the zeros are what it computed, not a file it never wrote.

The same argument applies to every all-zero directory above.

Consequence

Wherever a gate's admissibility test is gap > band, an all-zero artifact makes
that test unconditionally true and the assertion has no failure mode. #2839
measured exactly that for Glm4MoeLiteForCausalLM, where the artifact is
(8,16) int32, min 0 max 0, against a 500 mnat band.

This issue is about the ARTIFACT CLASS. It does not claim any particular gate is
wrong beyond #2839's, and it does not propose widening or narrowing a band. The
first thing it asks for is a re-capture on a host that has the checkpoints, with
the script's own diagnostic output preserved, so that whether the zeros come
from the script's use of prompt_logprobs or from the artifacts being written
without a real run can be settled by evidence rather than by argument.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with scripts/-neartie-gap.py and inspect the artifacts under tests/parity/goldens/; reproduce the reported zero gaps with numpy. Re-capture on a host with the checkpoints while preserving the script's diagnostic output, then determine whether the zeros come from prompt_logprobs or from artifacts being written without a real run.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, numpy, python
Domain
machine-learning, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.