mudler / mudler/vllm.cpp

gfx1151 survey: llama-cli ignored -no-cnv, so the end-to-end control arm spun the chat UI to 24 GB and timed out 4 of 4

Open
#2,935 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: BACKEND-GATE-ROCM-LLAMACPP

The survey of 2026-09-04 (#2921) lost its llamacli arm: 4 of 4 legs
rc=124
, the 25-minute timeout, and four output files of about 24 GB
each
on the share.

This is not a board fault and not a slow run. llama-cli ignored -no-cnv,
started its interactive chat UI, answered the prompt, and then printed a >
prompt in a loop against stdin redirected from /dev/null until the timeout
killed it.

The evidence

llamacli-r1.out, first bytes:

Loading model... |-\|/-\|/
...
build      : b0-unknown
model      : /tmp/rocm-strix-q4k/models/Qwen3.8-27B-Q4_K_M.gguf
ftype      : Q4_K - Medium

available commands:
  /exit or Ctrl+C     stop or exit
  /regen              regenerate the last response
...
> The capital of France is

Paris

[ Prompt: 117.1 t/s | Generation: 12.2 t/s ]

> 
> 
> 

and the remaining ~24 GB is that > line. The clock sampler agrees: mean
sclk 606 MHz and 0.2 percent busy across ~6,050 samples per leg, so
the board was idle for essentially the whole 25 minutes. The work finished in
the first few seconds.

What it cost, and what it did not

It did not affect the llama.cpp figure. That comes from the llamacpp arm,
a different binary (llama-bench, sha256 e2acbe26...f2c1) which completed 4
of 4 at rc=0.

It did cost the survey its like-for-like row. llamacli exists because
llama-bench -p 0 is a pure decode figure while the vLLM leg times a whole
completion, and the survey did not want to compare two different definitions.
Losing it leaves exactly that comparison unbacked by a llama.cpp measurement
taken through the ordinary request path.

The figure is not entirely lost: each leg printed its rate before it hung.

leg prompt t/s generation t/s
r1 117.1 12.2
r2 116.3 12.2
r3 116.7 12.2
r4 117.4 12.1

That is llama.cpp's own reported rate at one decimal place, self-reported rather
than harness-timed, and it agrees with llama-bench's 12.219. It is quoted in
the published page as corroboration and never as a measured leg.

Acceptance

  • The llamacli leg terminates on its own on this llama.cpp revision, whatever
    flag or stdin discipline that now takes, and its rc is 0.
  • Its output is bounded. A leg that writes 24 GB to a CIFS share is a harness
    defect even when the flag is right, so the leg gets an output cap or a pipe
    that closes.
  • The harness parses the generation rate from the leg rather than leaving it to
    be read out of a 24 GB file by hand.

Reproduction

  • Device strix:gpu0, gfx1151, ROCm 7.2.4, boot id
    a5bc8128-f6ad-4767-8614-6923f88032e1.
  • llama-cli -m <gguf> -p 'The capital of France is' -n 64 -ngl 99 --temp 0 -no-cnv --seed 1, stdin < /dev/null, under timeout 25m.
  • llama-cli sha256
    d563d9877adb49b4ae2c0a6c23e4019548ffc0c7e2d3a7c4339af3f636646a55, from the
    pinned b10451 tree, source manifest 56c26d15...f3cda at LC_ALL=C.
  • Raw logs: /mnt/nas_share/rc/strix-survey-2497/out/survey-20260904/.

Related: #2921, #2497.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the llama-cli reproduction command from the pinned b10451 tree and compare its -no-cnv and stdin behavior with the raw logs under /mnt/nas_share/rc/strix-survey-2497/out/survey-20260904/. Trace the llamacli survey leg and its timeout/output handling. Done means the leg exits with rc 0, output stays bounded, and the harness parses its generation rate.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
cli, performance, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.