A100 (sm_80) first run: crashes + wrong GDN output in fast paths
Open
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Env: A100-SXM4-80GB · CUDA 13.0 · commit f921062b
Build: add -DVLLM_CPP_CUDA_ARCHITECTURES=80 + -DCUTLASS_ENABLE_TOOLS=OFF (CUDA 13 otherwise fails in CUTLASS tools/library: duplicate sm_100f flags).
Result: 342/358 tests pass. 14 stable failures:
- Crashes (illegal memory access): GDN, MLA, FA2, and model/engine tests — test_qwen3_5_gdn_spec_routing, test_cuda_deepseek_v4, test_mla_attention_block, test_llama_embedding_fold, test_llm_engine, test_openai_api_server, test_minimax_h3
- Wrong output: GDN projection hash ≠ vLLM oracle (test_op_parity)
- Expected on sm_80 (GB10/Blackwell-gated): fp8/fp4 capability + sm120a-only op tests
Impact: GDN/MLA/FA2 models may crash or diverge on A100. Happy to bisect with compute-sanitizer.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the named stable failures, especially test_qwen3_5_gdn_spec_routing and test_op_parity, on the stated A100/CUDA 13.0 build; use compute-sanitizer as suggested to investigate illegal memory accesses. Compare the GDN projection with the vLLM oracle and verify the affected GDN, MLA, and FA2 tests no longer crash or diverge, while keeping the explicitly sm_80-gated failures separate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100