deepseek-ai / deepseek-ai/DeepGEMM
sm121a: `paged_mqa_logits` varlen path produces illegal memory access; uniform path does not scale past batch 1 at long context
- Dominant language
- Cuda
- Stars
- 7.8k
- Forks
- 1.3k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 3
Description
DeepGEMM 2.6.1 (as bundled in a vLLM main snapshot), GB10 / sm_121a (DGX Spark), 2-GPU TP via vLLM. Consumer of these kernels: vLLM's DeepSeek-Sparse-Attention indexer, model GLM-5.3-Flash (DSA geometry: kv_lora 512, index_topk 2048).
- The SM120 kernel family added for #317 (`sm120_fp8_paged_mqa_logits.cuh` + `sm120_paged_mqa_logits.cuh` scheduler) **works single-sequence** on sm_121a, verified up to 128K context with exact needle recall.
- With `kIsVarlen` engaged (2 sequences in the decode batch, ~50K tokens each) we get `cudaErrorIllegalAddress`. Same fault with batch shapes produced by a multimodal encoder even at batch 1. Faults reproduce 100% within seconds of the shape appearing.
- With the **uniform** path (batch 2, identical lengths, `next_n=2`) there is no fault, but throughput drops ~10x vs batch 1 (2.2 tok/s aggregate vs 23.4 single) - the kernel appears to serialise or oversubscribe on the 48-SM part at long context.
GB10 has 48 SMs and ~101 KB smem/SM, which several kernels in this family assume larger (cf. the `persistent_topk` 128 KB issue). vLLM-side, vllm-project/vllm#49896 documents a NaN-logits -> garbage-indices -> illegal-access chain in this kernel family on SM120 prefill; our decode/varlen faults may share that root. If useful I can run instrumented builds or provide the exact tensor shapes from vLLM's dump on crash.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with sm120_fp8_paged_mqa_logits.cuh and the sm120_paged_mqa_logits.cuh scheduler, comparing the kIsVarlen and uniform paths. Reproduce the reported two-sequence, long-context shape on sm_121a and inspect the related persistent_topk 128 KB assumption and vLLM issue #49896. Done means the varlen path no longer produces illegal accesses and the uniform path avoids the reported long-context throughput collapse.
Written by the indexing model from the issue text.
Assessment
- Domain
- ai-infra-agents, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100