mudler / mudler/vllm.cpp

port: HY V4 is a post-pin architecture no row owns, and its two shared-file halves are pre-pin holes rather than the distance

Open Beginner friendly
#2,735 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: none exists. No row in .agents/roadmap_v1.md or any *-matrix.md owns HY V4. The nearest existing rows are MODEL-TEXT-hy-v3-hyv3-for-causal-lm (.agents/model-matrix.md:259) and MODEL-SPEC-hy-v3-mtp-hyv3-mtp (.agents/model-matrix.md:581), which own the V3 family, not this one. grep -rn "hy_v4" .agents/ returns 2 hits and grep -rn "HYV4" .agents/ returns 2, all four inside sync reports (2026-09-01-cdefd9d.md:137,1072, 2026-09-01-adebc41.md:267,669) — which is where the absence is recorded and nowhere else. Positive control through the identical form and scope: hy_v3 returns 21 hits across .agents/. This issue does not invent a row.

Found by wave PORTQ-7 (#2717) re-deriving PORT-NOW entry [278], upstream b2f685834a vllm#54160. Nothing was executed.

Read this first: the two halves the queue line emphasises are NOT the gap

The record line for this entry leads with the shared-file edits, "besides the new (inventory-worthy) HY V4 model". Both of those halves are surface-absent here, and both rest on pre-pin holes:

  • deepseek_v2.py::_try_load_fp8_indexer_wk (E8M0/MXFP8 indexer scale decoding). This tree never reaches that shape. DeepseekV2ForCausalLM refuses V3.2 DSA-indexer checkpoints by name at src/vllm/model_executor/models/deepseek_v2_weights.cpp:370-375; the dots3-note DSA model refuses the blockwise-FP8 arm by name at src/vllm/model_executor/models/dots3_note_device.cpp:810-824; and the DSA loaders that do run take GGUF (src/vllm/model_executor/models/glm_moe_dsa_loader.cpp:464). _try_load_fp8_indexer_wk existed at 5559679229 (deepseek_v2.py:822, called at :1588) and was never ported.
    Worth recording for whoever eventually ports it: this tree's one FP8-block-with-scale reader already does both things the commit adds — it requires the scale dtype to be E8M0 explicitly (RequireDtype(s, "F8_E8M0", sname)) and derives the block grid per dimension, at src/vllm/model_executor/models/deepseek_v4_weights.cpp:550-578. The fix's shape is already this tree's convention, just on a different tensor family.
  • speculator.py's hc_mult widening. This tree's DSv4 drafter takes the opposite approach: dspark::StreamMeanTap collapses [T, hc_mult, hidden] to [T, hidden] by mean over the stream axis (include/vllm/model_executor/models/deepseek_v4_dspark.h:22-40), so drafter buffers are never widened and cannot be 4x too wide for an iHC target. get_mtp_target_hidden_states -> 0; include/vllm/v1/spec_decode/draft_model_proposer.h:5 records pass_hidden_states_to_model=False.

A scheduling warning for the first half. 28158b2fc3 vllm#48886 is an earlier in-range commit on the same function — it added the scale_inv.ndim == 1 per-channel branch that b2f685834a then generalises. Porting b2f685834a alone would land half a function.

The gap is the model

The whole HY V4 architecture is absent: attention, FlashMLA-sparse, iHC, MoE and MTP under vllm/models/hy_v4/nvidia/, plus transformers_utils/configs/hy_v4.py, the registry entry, the reasoning parser and the tool parser. Roughly 3200 upstream product lines. Every one of those paths is absent at the pin as well (git cat-file -e 5559679229:vllm/models/hy_v4/nvidia/model.py fails), so this is a post-pin architecture, not a missed forward-port.

Size here: a full model port, comparable to the qwen4_exp campaign. That is why this issue exists to own the gap, not to schedule it.

What this issue asks for

A decision, not an implementation: either open a MODEL-* row for HY V4 in .agents/model-matrix.md beside the existing V3 rows, or record HY V4 under ## Owed in a spec that names it. Right now the only trace of it in this repository is four lines inside two sync reports, which is not an owner.

Note also that HY V3's tool parser is ported here (src/vllm/entrypoints/openai/tool_parsers/hy_v3.cpp) and has its own separate gap, #2727, from PORT-NOW entry [258]. That is the V3 parser, not this.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the HY V4 references in .agents/model-matrix.md, .agents/roadmap_v1.md, and the two 2026-09-01 sync reports, then verify the absence with the searches described in the issue. Done means recording HY V4 as an owned MODEL-* row beside the V3 rows or adding it under ## Owed in a specification; no model implementation is requested.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.