Qwen3.8-27B Q4_K_M is not token-exact vs llama.cpp b10451 on the CPU tier: 5 of 6 prompts diverge at rank-2 under 0.18 logits

Row: QUANT-QWEN38-27B-GGUF-ARM

The Qwen3.8-27B-Q4_K_M.gguf arm is not token-exact against the pinned
llama.cpp b10451, and this is not a ROCm defect — it was measured on
thor:gpu0, an NVIDIA device. Until it passes, AGENTS.md §Gates admits no
speed or memory result from this arm on any backend.

What is recorded

docs/bench-evidence/qwen38-27b-q4km-token-gate-20260823.md, artifact
gguf_sha256 7e78da5d7e3ae28d178121f58646953305f3e5bd3cb46f4a75584e8b6c6fe169,
17,106,775,008 B:

• Tokenizer: exact, 6 of 6 prompts.
• Generation: diverges on 5 of 6. TOKEN_GATE=FAIL.
• Every divergence is a rank-2 loss under 0.18 logits, against absolute
  logits of 15.9 to 22.6, with 282 of 288 steps at rank 1.

So the arm is arithmetically close and structurally right. A wiring defect would
put our token far down the oracle's ranking repeatedly, and would not leave 282
of 288 steps at rank 1. This is a precision difference in the quantized compute
path.

Why it cannot be waived

The ratified distributional band is admitted only where the oracle's greedy
decode is non-deterministic. This oracle's is deterministic: it reproduced
#857's output byte for byte
from a different build. The band's premise fails and the earlier run explicitly
did not reach for it. A 0.058-logit loss is a small margin, not the exact fp32
tie #910 adjudicated.

Why this issue exists now

It is the correctness blocker under a standing goal: complete ROCm support for
Qwen3.8-27B on Strix Halo at our benchmark targets. Performance work on this arm
is inadmissible until this passes, on ROCm and on CUDA alike. It also silently
invalidated a measurement once already — see
#2497, where a paired gfx1151
run produced a ratio that had to be retracted because nobody checked this
precondition first.

Nothing owned this. The QUANT-QWEN38-27B-GGUF-ARM row records the divergence
as a property of the arm and leaves it at PARTIAL; no issue tracked closing it.

Scope

Find and remove the precision difference, or establish on evidence that it
cannot be removed and what that means for the arm.

Candidate directions, none yet tested:

• Accumulator width and reduction order in the Q4_K/Q6_K dot products. A
  double accumulator read through float hides reduction-order differences, and a
  bf16 store absorbs them; either can leave a sub-0.2-logit bias that only shows
  at a rank-1/rank-2 boundary.
• The Q8_K activation quantizer (QuantizeQ8KK), which is shared by the
  whole k-quant path and is separately implicated in throughput
  (#1876).
• Per-superblock scale handling and whether our d/dmin arithmetic
  matches the oracle's exactly rather than approximately.
• Read the oracle's ggml-quants.c dot products at b10451 and compare the
  executing chain, not the source shape.

Acceptance

• TOKEN_GATE=PASS on this artifact against b10451, 6 of 6 prompts, on CUDA.
• The same gate run on ROCm once
  #2511 lets that board
  complete a generation.
• The cause named against specific arithmetic, with a red-before reproduction.
• If any residual divergence remains, it is quantified and its admissibility
  argued explicitly rather than assumed.

Related

• #2497 — the retracted gfx1151 measurement this blocked.
• #2511 — the gfx1151 hang, which blocks running this gate on ROCm at all.
• #1876, #487, #1400, #1863 — throughput work on the same path, all of which
  stays inadmissible until this passes.

Start with the recorded artifact and TOKEN_GATE failure, then compare the executing Q4_K/Q6_K dot-product chain with the oracle's ggml-quants.c at b10451. Inspect accumulator width, reduction order, QuantizeQ8KK, and per-superblock d/dmin handling, using a red-before reproduction. Done means CUDA and then ROCm both pass the six-prompt gate, or residual divergence is quantified and explicitly justified.