mudler / mudler/vllm.cpp

The hybrid-placement speed axis is untracked: its campaign issue #149 was deleted, and the fleet has no device that can measure it

Open
#3,013 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: ENG-HYBRID-PLACEMENT

What

ENG-HYBRID-PLACEMENT has two waves that are blocked on hardware, and the issue
that owned them no longer exists.

.agents/specs/hybrid-placement.md cites #149 five times — including as the
row's own campaign issue in its | Row ID | line — and #149 returns 404 to an
unauthenticated reader while a known-live issue returns 200 from the same probe.
#147, cited beside it as the multi-GPU half, is also 404. So the row's records
point at two deleted owners, and its blocked work has had no tracker.

What is actually owed

W0 — the measured DDR:PCIe ratio and the per-MoE-layer round-trip cost.
## The honest bandwidth math in the spec decides whether this row is worth
anything at all, and every number in it comes from published link rates rather
than from measurement. The spec says so itself: the row is worthless if the real
ratio is below ~1.5x, "and that is a plausible outcome on a rig with slow
single-rank memory."

W5 — the speed floor against llama.cpp -ncmoe at pin b10451. The
correctness side is done and measured: NMSE 5.239e-06 for the cpu_moe arm and
3.569e-05 for --fit, on GB10/sm_121 against
Qwen3.8-Flash-Next-UD-IQ1_S. What has never been measured is whether placement
is FASTER than not placing.

Why it is blocked, precisely

Placement moves activations over a bus to buy device memory. On a unified-memory
part there is no bus to cross and no memory to buy, so the capability is inert by
construction and the axis cannot be measured — not "is hard to measure".

rc devices lists dgx:gpu0, orin:gpu0 and thor:gpu0. All three are
integrated parts. The fleet contains no device that can measure this row's
speed axis.

What would unblock it

One discrete-GPU host with a PCIe-attached accelerator and host DRAM. Nothing
else about the row changes: the seam, the plan resolver, the --fit resolver and
the gates all exist and are exercised on CPU today
(test_placed_moe_roundtrip.cpp executes the placed branch with no GPU at all).

Until then

No speed claim may cite this row, and the bandwidth table in the spec stays
marked as an assumption. That is the state today and this issue exists so it is
visible rather than implied by a dead link.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read .agents/specs/hybrid-placement.md, especially “The honest bandwidth math,” then run rc devices to confirm the available hardware and review test_placed_moe_roundtrip.cpp for the existing placed branch. Done requires a discrete-GPU host with PCIe-attached acceleration and host DRAM to measure the DDR:PCIe ratio, per-layer round-trip cost, and W5 speed floor against llama.cpp -ncmoe at pin b10451.

Written by the indexing model from the issue text.

Assessment

Domain
performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.