llama.cpp b10451 built for HIP emits degenerate output at n_gpu_layers=0 on gfx1151's host, while n_gpu_layers=99 is correct
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: BACKEND-ROCM
Measured 2026-09-02 on strix:gpu0 (rc job 2644106f-3123-4c18-8d42-d4db11fe62e8,
worker rc-worker-lcjhd, x86-64, AMD RYZEN AI MAX+ 395) while running the
gfx1151 Q4_K_M token gate for
#2546.
One stock b10451 binary, built with -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151,
loaded the recorded Qwen3.8-27B-Q4_K_M.gguf (sha256
7e78da5d7e3ae28d178121f58646953305f3e5bd3cb46f4a75584e8b6c6fe169) twice in one
job, greedy, 48 tokens, identical prompts. The only difference is
n_gpu_layers.
n_gpu_layers = 99 is coherent and correct. Its ids agree with the
2026-08-23 aarch64 run of the same oracle on 3 of 6 prompts exactly and differ on
3 by one token each.
n_gpu_layers = 0 is degenerate. It emits one repeated token for the whole
48-token completion on 5 of 6 prompts:
prompt 0: n=48 distinct=1 first5=[286, 286, 286, 286, 286]
prompt 1: n=48 distinct=3 first5=[90, 13, 641, 641, 641]
prompt 2: n=48 distinct=1 first5=[383, 383, 383, 383, 383]
prompt 3: n=48 distinct=1 first5=[393, 393, 393, 393, 393]
prompt 4: n=48 distinct=1 first5=[96229, 96229, ...]
prompt 5: n=48 distinct=1 first5=[12512, 12512, ...]
system_info for both legs:
ROCm : NO_VMM = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1,
use_extra_bufts = 1, devices ROCm0 Radeon 8060S Graphics and
CPU AMD RYZEN AI MAX+ 395 w/ Radeon 8060S.
Why this matters beyond a broken leg
That leg existed to be a bridge: the CPU tier's recorded oracle ids came from
aarch64, so an x86-64 CPU leg from the same build was meant to separate the host
term from the kernel-path term when comparing tiers. It cannot serve that purpose
and was not used. The comparison in #2546 rests on the HIP leg against the
recorded aarch64 leg instead.
It also means no CPU-tier number should be taken from a HIP-built llama.cpp on
this host until this is explained.
Not yet discriminated
This is recorded as an observation, not a diagnosis. The harness is the
2026-08-23 oracle_tokens.cpp, unchanged except for reading n_gpu_layers and
use_extra_bufts from the environment, and the identical code path is coherent
at ngl=99, so the harness is not obviously implicated. Three candidates, in the
order they are cheap to test:
use_extra_bufts=1(the AVX512 Q4_K repack path) on this CPU. Re-run the same
leg withORACLE_EXTRA_BUFTS=0; if it is coherent, the repack path is the
cause.- A mixed-backend scheduling defect at
ngl=0in a HIP-enabled build, where the
ROCm backend is registered but no layer is offloaded. Test with a CPU-only
build of the same pin on the same host. - Board or driver state left behind by the preceding HIP leg in the same
process lifetime. Test by running thengl=0leg first in a fresh process.
Whoever picks this up should run 1 and 2 before anything else; they are one lease
between them and they partition the space.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the unchanged 2026-08-23 oracle_tokens.cpp harness and reproduce the gfx1151 ngl=0 leg. Run the two requested checks first: repeat with ORACLE_EXTRA_BUFTS=0, then compare a CPU-only build on the same host. Done means identifying which candidate explains the degenerate output and recording whether the coherent ngl=99 behavior is preserved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100