mudler / mudler/vllm.cpp

Captured-pair gates: score fresh runs against an oracle-side-only top-K golden instead of the self-anchor

Open
#3,160 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: BACKEND-TENSTORRENT-KEEPQUANT

Problem

The captured-pair gates (the qwen3.5 Q4_K_M TT vehicle today; the same
ladder shape in the qwen3/mistral gates) pin two functions of OUR engine's
numerics: the exact ids anchor (self-match REQUIRE) and a per-capture
teacher-forced gap golden. Every ULP-level shift — a tt-metal pin bump, a
kernel change — re-rolls the ~51/256 near-tie cells and turns the gate red,
forcing a golden refresh. The refresh itself is where the process failed on
2026-09-12: an off-recipe re-derivation wrote f32-grain gaps that
manufactured four band violations and nearly drove a kNearTieMnats
widening (recorded in .agents/specs/tenstorrent-keepquant.md, Evidence).
The pinned-recipe truth was max gap 250 mnats, 0 cells above band — the
engine was fine; the refresh path was not.

Proposed shape

Commit an oracle-side-only golden: per prompt cell, the pinned oracle's
top-K logprobs under the ORACLE's own greedy prefix — stable under our code
changes, so it never re-captures for engine-side reasons. The gate scores
the fresh engine run against the committed top-K up to the first
divergence and fails only a confident divergence (gap above band, or our
token outside top-K). Near-tie flips pass with a recorded margin. The
self-anchor exact REQUIRE becomes an advisory MESSAGE recording the
re-roll. At the decision cell this is exactly the bar the gap script's own
header declares ("is our token, given our exact prefix, one the oracle
cannot confidently separate"); post-flip cascade cells are cross-context
and uninformative today anyway.

Why not now

The verified pair plus the provenance-hardened re-derivation path
(scripts/qwen35-q4km-neartie-gap.sh) lands in #2959's flow. This
redesign is a semantic checker change: it needs its own spec, a
red-before demonstration (a ULP re-roll failing today's anchor),
mutation evidence that a real divergence still fails, and fresh review.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with .agents/specs/tenstorrent-keepquant.md and scripts/qwen35-q4km-neartie-gap.sh, then review the verified-pair and #2959 flow. Define the checker change with its own spec, demonstrate a red-before ULP re-roll, and provide mutation evidence that a real divergence still fails before review.

Written by the indexing model from the issue text.

Assessment

Tech stack
bash, cpp
Domain
backend, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.