mudler / mudler/vllm.cpp

The block-wise FP8 GEMM owner #1189 is a deleted issue, and a shipped refusal string hands it to operators

Open
#3,007 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: -

#1189 — cited 21 times in the issues/1189 URL form, and named as the owner of
the block-wise FP8 GEMM work ("milestone M5") — does not exist. It is a 404
to an authenticated caller and to an anonymous one alike, while #2881 answers
200 on the same probe. That two-sided check is what distinguishes a deleted
issue from one merely hidden from an unauthenticated reader, and it is the same
probe that established #699 in #2924.

Why this one is worse than #699

#1189 is not only in records. It is baked into a shipped runtime string:

// src/vllm/model_executor/layers/quantization/fp8_block_quant.cpp:67
const char* kIssue = "https://github.com/mudler/vllm.cpp/issues/1189";

RefuseUnrunnableFp8BlockWeight interpolates that constant, so every operator
who hits a block-wise FP8 refusal is handed a dead link by the running binary
and told it is where the rest is owed. The refusal is otherwise exemplary — it
names the projection, the device, and the cutlass-fp8 arch cell — which makes
the dead pointer the only part a reader cannot act on.

It is also the sole unblocking condition of a ## Owed entry added by
#2947 on
MODEL-MM-dots3-note-dots3-note-for-causal-lm: the vision FP8 arm refuses on
every CUDA arch outside 12.0a,12.1a, and what unblocks it is "a block-scaled
FP8 GEMM registered for those arches — #1189 milestone M5". That entry survives
the dead number, because it states the condition independently as widening
VT_CUTLASS_FP8_ARCHS past 12.0a,12.1a, or an equivalent arm
— but the
number is what a reader follows first.

Where it is cited

21 URL-form citations. Beyond the runtime string above, the live pointers are in
.agents/model-matrix.md, .agents/engine-matrix.md, .agents/parity-ledger.md,
.agents/benchmark-record.md, .agents/specs/dflash-spec-decode.md,
.agents/specs/fix-probe-cannot-say-no.md and
src/vllm/model_executor/models/dots3_note_vision.h. The remainder are in
.agents/completed/ — state events and migration manifests, which are archived
evidence of what was true when written and should not be repointed, per
AGENTS.md §Records ("Never delete evidence").

What this asks for

  1. Decide what actually owns block-wise FP8 GEMM milestone M5 now — an existing
    issue, or a new one filed for it. The work is real and unfinished regardless
    of what happened to the number.
  2. Repoint the live pointers, starting with kIssue, because that one is handed
    to users at runtime.
  3. Leave the .agents/completed/ citations alone.

Not urgent for correctness: no gate reads the number, and every refusal that
cites it still names the missing capability in prose. This is a broken pointer
in operator-facing text, of the same class as #2924 and found the same way.

Owed to: whoever owns the FP8 block GEMM lane. Filed from #2947's delta review,
which caught it because that PR removed a dead #699 citation and added three
dead #1189 ones a line away.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with AGENTS.md §Records and src/vllm/model_executor/layers/quantization/fp8_block_quant.cpp, then inspect the live references in the listed .agents files and dots3_note_vision.h. Determine whether an existing issue owns block-wise FP8 GEMM or whether a new issue is needed; update live pointers, including kIssue, while leaving .agents/completed/ citations unchanged. Done means the runtime refusal and live records point to the current owner.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
documentation, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.