Qwen3.8-27B Q4_K_M: p4/14 is the CPU tier diverging across ARCHITECTURES, not ROCm; #2590 narrows to two steps
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: QUANT-QWEN38-27B-GGUF-ARM
#2590 was opened owning three steps at which our ROCm and CPU tiers emit a
different token from an identical prefix. Run on ONE host, in one lease, from one
libvllm.so, only two of them reproduce.
| step | recorded in #2590 | measured on one host (strix:gpu0, rc job 5af7552a-5fba-4e51-92d8-f4cdfd3b21ca) |
|---|---|---|
| p1/45 | ROCm 303, CPU 1521 | ROCm 303, CPU 1521 — reproduces |
| p3/45 | ROCm 25, CPU 393 | ROCm 25, CPU 393 — reproduces |
| p4/14 | ROCm 22486, CPU 4593 | both tiers 22486; all 48 ids EQUAL |
#2590's A vs D column compared a gfx1151 ROCm run on x86-64 against a
CPU-tier run on thor, aarch64, so it crossed an architecture as well as a
tier. With both arms on x86-64 our CPU tier emits 22486 at p4/14, agreeing with
the ROCm tier, with the HIP oracle and with the aarch64 oracle.
So p4/14 is the CPU tier's own architecture dependence, not a ROCm-local
term. The token the aarch64 CPU tier emitted there (4593) is one no other
side emits. This issue owns that, and #2590 is narrowed to two steps.
What is already known
- The step is a near-tie: the HIP oracle's
top1 - top2gap at p4/14 is
0.092752, inside the 19-step population whose gap is below 0.20. - The x86-64 CPU tier is bit-reproducible per layer against itself, and so is the
ROCm tier: the run-to-run floor measured here is exactly 0 over 9360 compared
per-layer rows. - The cross-tier hidden-state divergence at layer 63 is 2.6-2.8% relative, which
is 1.03 to 1.11 times what two independent bf16 residual streams must produce.
A near-tie under roughly half a nat is decided by that, so a divergence at a
0.0928 gap needs no further cause than a different reduction order. - The one candidate that is NOT explained by that is why aarch64 and x86-64
differ from each other, which is what this issue is for.QuantRepackForDevice
isdev == kCPUonly and the aarch64 lane carries theblock_q8_0x4interleave
(#2406); x86-64 does not repack Q4_K either, so the repack is a candidate and
not a conclusion.
The next traceable step
Run the same per-layer hidden-state comparison this measurement built
(scripts/tier-hidden-delta.py, scripts/tier-delta-attribution.py,
VT_DUMP_ACT) with the two arms being aarch64 CPU and x86-64 CPU on the same
artifact and the same prompt, and read the per-layer profile the same way. The
instrument is committed and needs no GPU on either side.
Not admissible from this
No speed, latency or memory number. The arm's declared token gate is FAIL and
AGENTS.md §Gates admits none from it.
Evidence:
docs/bench-evidence/qwen38-27b-q4km-tier-divergence-bisect-20260902.md.
Spec: .agents/specs/rocm-tier-hidden-state-bisect.md.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with scripts/tier-hidden-delta.py and scripts/tier-delta-attribution.py, using VT_DUMP_ACT with the same artifact and prompt on aarch64 CPU and x86-64 CPU. Compare the per-layer hidden-state profiles and trace where the architectures diverge; done means documenting the attributable divergence and whether the block_q8_0x4 repack remains a candidate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100