mudler / mudler/vllm.cpp

VT_DUMP_IDS=1 turns the qwen3 paged-engine anchor REQUIRE off and the PASS verdict reads only the committed goldens, so a dump-mode run can print PASS while comparing nothing

Open
#2,671 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: BACKEND-TENSTORRENT-HOST-FREE-FORWARD

Symptom

VT_DUMP_IDS=1 makes the qwen3 paged-engine golden battery skip its anchor
REQUIRE (tests/parity/test_qwen3_paged_engine.cpp:339,
anchor_ok = dump || first_div < 0). The PASS verdict reads only the
committed golden pair, so a dump-mode run prints 16/16 prompts PASS
whenever the committed goldens are internally consistent, even when every
freshly decoded token diverges from the anchor. An operator who re-runs the
battery in dump mode gets a green line that proves nothing about the current
build.

This went live on 2026-09-02 during the #1625 chase: a dump-mode run printed
16/16 PASS on a build whose non-dump run reds 14 of 16 prompts at the
boundary.

Why the skip exists and what to keep

The skip is intentional for re-capture: a stale anchor must not block a
golden refresh, and qwen3-neartie-gap.py depends on this path. The trap is
the verdict line, which does not say the run was dump-mode and does not say
the anchor comparison was off.

Repair direction

Make dump-mode loud: print RE-CAPTURE MODE: anchor comparison skipped on
the verdict line and report the count of skipped anchors, so a PASS in
re-capture mode is distinguishable from a PASS that compared anything. The
row's golden-refresh path through qwen3-neartie-gap.py must keep working.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in tests/parity/test_qwen3_paged_engine.cpp at the anchor check near line 339 and inspect the dump-mode verdict path. Run the qwen3 paged-engine battery with VT_DUMP_IDS=1, then verify that the verdict identifies re-capture mode and reports skipped anchors while qwen3-neartie-gap.py still works for golden refreshes.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.