mudler / mudler/vllm.cpp

GLM-5.3 non-flash on gfx1151: the forward dumps core, so no token has been observed since #2511

Open
#2,965 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: BACKEND-ROCM

Spec: .agents/specs/rocm-mla-attention-nonflash.md

What is settled

#2926 landed native kROCM kMlaPrefillAttention and kMlaDecodeAttention
(src/vt/rocm/rocm_mla_attn.hip, merged at 967c1906f). They are registered,
gated against the CPU oracle, and REACHED on the real 201.83 GiB
unsloth/GLM-5.3-GGUF UD-IQ1_S artifact on strix:gpu0
(op=33 device=5 selected=vt-native registered=1). All five numeric
guarantees are killed by the gate rather than by the compiler — rc job
a77aa104 re-expressed the three that -Werror,-Wunused-* had killed, and the
test caught all three.

So the DENSE-step refusal is gone. That is not the same as generating.

What is open

No token has been observed on any tree carrying 6b97a6800 (#2511).

The leading explanation was the allocator: #2511 narrowed managed allocation to
PageableMemoryAccess == 1, gfx1151 reports 0, and the run that last emitted
Paris, which is predates it. The prediction was that a leg under
VT_ROCM_MANAGED_ALLOC=1 — which the refusal message itself names — would
generate.

That prediction is refuted. rc job a77aa104 (strix:gpu0, tree
56fd248e9, 3 h timeout) ran exactly that leg:

LEG[managed] rc=0 wall=4118s
[  4118s] [vt op-provider] op=4 device=5 selected=vt-native ...
[  4118s] vllm.cpp: Asynchronous scheduling is disabled (max_concurrent_batches=1)
[  4118s] timeout: the monitored command dumped core

The load completed and the forward had begun; the process then died on a
core-producing signal at 4118 s, well inside its 10800 s budget. The rc=0 is
not that process's status — the runner read $? after
( … | awk … ) > log, which is awk's. The verdict is the timeout line.

So the crash is inside the forward, and the managed allocator is not the
way back to a token. That is consistent with #2511's own measurement of 17 GPU
faults in 21 managed legs against 0 in 21 without, and it is why that flag
stays a diagnostic lever and never a shipping default.

What is owed

  1. The default-allocator control leg. Job a77aa104's LEG B never ran: the
    CIFS share dropped after leg A (error reading input file: Host is down
    bash reads a script lazily by byte offset and the script lived on
    /workspace), and the job idled to its 8 h max_runtime kill. There is
    still no A/B, so the allocator is not isolated from the kernels. A rerun
    stages its own script into /tmp first, as it already stages the source.

  2. Separate a board fault from a kernel fault. #2546 measured 12/12 GPU
    resets on this board under a gate-sized native run, so "the forward crashed"
    does not yet name which. A core file, dmesg, or an AMD_SERIALIZE_KERNEL=3
    leg is the next traceable step.

  3. The ~41 min load under plain hipMalloc is still an observation, not a
    measurement.
    a77aa104 carried a per-line timestamper meant to settle it
    and the timestamper did not work: vllm-cli block-buffers behind a pipe, so
    every line — the first line of the load included — reached awk at flush and
    was stamped [ 4118s]. Defeat the buffering at the source (stdbuf -oL, or
    a pty) rather than stamping downstream of it.

  4. kDsaIndexerLogits + kDsaTopkSelect on ROCm (#2715 W2, closed with the
    W1 half). Not generation-blocking on a dense step; a SPARSE step — a prompt
    past index_topk, where run_indexer turns true
    (mla_attention.cpp:493-494) — still refuses. A benchmark workload drives
    sparse steps
    , so this blocks any Strix speed number even once generation
    returns.

Why this matters

The ROCm GLM-5.3 speed axis stays VOID while this is open, and it is the
blocker on the "at least as fast as vLLM on Strix Halo" goal for this model.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with .agents/specs/rocm-mla-attention-nonflash.md and src/vt/rocm/rocm_mla_attn.hip. Rerun the default-allocator control with the script staged in /tmp, then collect a core file, dmesg, or an AMD_SERIALIZE_KERNEL=3 trace to separate board and kernel faults. Check mla_attention.cpp:493-494 for the sparse-step path and fix the forward crash and remaining sparse-step refusal so generation and benchmarking can complete.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.