Target-only Qwen3.8 W4 sampled decode selects mixed-dtype packed GDN
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Native Windows CUDA target-only decode for the Qwen3.8-27B W4A16 gate model fails before producing a sampled token when temperature is 0.7 and top-p is 0.8. The production path reaches vt::GdnPackedDecode with mixed_qkv/a/b/out not sharing one FP16/BF16/F32 dtype and throws at src/vt/ops.cpp:2496. The same target model runs through speculative DFlash2 verification because that step takes a different GDN recurrence route. Reproduction evidence: J:\vllmcpp-dflash2-w4-runs\73-target-only-repro-t07-p08-1x4. Owning row: SPEC-REJECTION. The fix must preserve the packed-decode dtype predictor/producer invariant and add a target-only production regression test.\n\nFOLLOWING_AGENTS_PROTOCOL\n\nFollowing-Agents-Protocol: true\nAI-Assisted: true\nAssisted-by: AGENT:GPT-5 [CODEX]
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the mixed-dtype failure in src/vt/ops.cpp:2496 and use the reproduction evidence at J:\vllmcpp-dflash2-w4-runs\73-target-only-repro-t07-p08-1x4 to trace the target-only production path into vt::GdnPackedDecode. Compare the packed-decode dtype predictor and producer, then add a target-only production regression test covering the Qwen3.8-27B W4A16 sampled decode case. Done means the path preserves the shared dtype invariant and produces a sampled token without the rejection.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100