A ModelOpt checkpoint declaring NVFP4 W4A4 runs our W4A16 arm silently: RadixArk/Qwen3.8-27B-NVFP4 ships input_scale on all 193
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: QUANT-QWEN38-27B-NVFP4-ARM
A ModelOpt checkpoint that declares quant_algo: "NVFP4" and ships an input_scale on every NVFP4 module runs this tree's W4A16 weight-only arm, silently, and says nothing.
RadixArk/Qwen3.8-27B-NVFP4 @ 554ebba9b5f1b79dc11246341960360e6ef05ef4 is the target pangoleen/qwen3.8-27b-dgx-spark-dflash2 serves under SGLang, and it is the artifact r0b0tlab/Qwen3.8-27B-NVFP4-MTP-sm121 was derived from. Read from both config.json files at the pinned revisions, the two differ in exactly two ways that matter:
RadixArk @ 554ebba9 |
r0b0tlab @ 36f717a2 |
|
|---|---|---|
all 193 quantized_layers MLP entries |
{"quant_algo": "NVFP4", "group_size": 16} |
{"quant_algo": "W4A16_NVFP4", "group_size": 16} |
config_groups.group_1.input_activations |
{dynamic: false, num_bits: 4, type: float, group_size: 16} |
null |
<proj>.input_scale on an NVFP4 module |
192 MLP + lm_head |
zero |
kv_cache_scheme |
{dynamic: false, num_bits: 8, type: float} |
absent |
Everything else agrees: the same 208 FP8 + 193 NVFP4 split over the same 401 module names, and shard 1's committed 970-tensor fixture tests/vllm/models/qwen38_27b_modelopt_mtp_s1_manifest.inc is a subset of RadixArk's 2194 tensors with zero dtype or shape disagreements. So r0b0tlab is RadixArk re-split, with the MTP shard separated, the input_scale tensors stripped and the algorithm relabelled weight-only.
What this tree does with the RadixArk spelling. Both whole-checkpoint refusal gates answer "" — compiled against 3047871581bc55a0ab1a44006421bbe02698d5b8 and run on the real config.json plus the real 2194 tensor names:
compressed-tensors refusal: []
IsMixedPrecision: 1
modelopt refusal: []
layers::modelopt::Refusal treats QuantAlgo::kNvfp4 and QuantAlgo::kW4A16Nvfp4 identically — one shared case accepting any module that ships an NVFP4 spelling — so the declaration passes its own cross-check. Routing is then done by the tensor-name probe, and VT_MODELOPT_W4A4 defaults to 0 because consuming input_scale produced incoherent text on nvidia/Qwen3.6-27B-NVFP4 (docs/ENVIRONMENT.md). The result is that we execute the weight-only W4A16 arm against a checkpoint whose producer declared static fp4 activations, and no line of output says so.
This is the live half of what the tree records as #1597 in .agents/completed/issue-index.md (that number does not resolve on the forge from this session, so treat its state as REMOTE_UNVERIFIED rather than closed), on a new artifact and with a consequence attached: it is now an unmatched axis in a published comparison rather than a latent one. It is also not a "wrong bytes" defect — the weight bytes swept per decode step are identical either way, so the roofline arithmetic in .agents/specs/bench-qwen38-27b-nvfp4-matched.md §2 is unaffected. What differs is the activation path and the numerics, and a token gate cannot see either, because W4A16 of a W4A4 checkpoint is numerically plausible.
What is wanted, in order of value.
- Say it. When the declaration is
NVFP4and the executed arm is W4A16, print one line naming both. Visible debt beats silent divergence, and it costs nothing. - Decide whether
VT_MODELOPT_W4A4should be reachable on this checkpoint at all, given it is recorded as producing incoherent text on the 27B gate model. If our W4A4 path is wrong, that is a correctness gap on a shipped format, not a lever that happens to be off. - The larger change #1597 already scoped (see the caveat above on its state): route by the declaration where one exists, keep the name probe where nothing is declared, and pay the same-binary A/B on
nvidia/Qwen3.6-27B-NVFP4that it owes, because the point of the change is that some module's arm moves.
The checkpoint is staged and pinned; its weights rows are in docs/USAGE.md with this change. Owed under ## Owed in .agents/specs/bench-qwen38-27b-nvfp4-matched.md.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the modelopt refusal and tensor-name routing code that selects the W4A16 or W4A4 arm, using the pinned config.json values and tests/vllm/models/qwen38_27b_modelopt_mtp_s1_manifest.inc as evidence. Read docs/ENVIRONMENT.md and the referenced .agents specs to understand the existing gate and owed comparison. Done should make the selected arm visible and establish how declaration-based routing and the same-binary A/B should be evaluated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100