deepseek-ai / deepseek-ai/DeepGEMM
[Regression] 8b1392b removes SM12x pure-fp8 1d1d kernels and aliases fp8_gemm_nt to the fp4 dispatcher (silent corruption on GB10)
- Dominant language
- Cuda
- Stars
- 7.8k
- Forks
- 1.3k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 3
Description
## Summary
Between `a6b593d` and `8b1392b` on `nv_dev`, the SM12x pure-fp8 path regressed.
Verified diff (`git diff a6b593d..8b1392b`):
- **Removed**: `csrc/jit_kernels/impls/sm100_fp8_gemm_1d1d.hpp` (−416) and
`deep_gemm/include/deep_gemm/impls/sm100_fp8_gemm_1d1d.cuh` (−567) — the
pure-fp8 1d1d kernels.
- `fp8_fp4_mqa_logits` dispatch rewritten: generic `smxx_fp8_mqa_logits`
→ per-arch `sm90_fp8_mqa_logits` / `sm100_mqa_logits` / `sm120_mqa_logits`.
- NOTE (correction of the earlier draft): the `fp8_gemm_nt = fp8_fp4_gemm_nt`
alias is **not** new — it exists in `a6b593d` already (`gemm.hpp:792`).
The regression is the 1d1d kernel removal, not the alias.
## Impact (measured on 2x DGX Spark, GB10 / sm_121a)
With `--linear-backend deep_gemm` (DeepSeek-V4-Flash FP8 linear + MLA),
fp8xfp8 inputs on SM12x run the combined `sm120_fp8_fp4_gemm_1d1d` kernel,
which misreads fp8 weights as fp4: **silent numerical corruption**. Greedy
France output degenerates (`' Septy Septy…'`) and DSpark draft acceptance
collapses, dropping decode from ~25.8 tok/s (b12x linear baseline) to
~4.4 tok/s.
`a6b593d` (the last good commit, frozen by eugr/spark-vllm-docker and used
by vLLM v0.25.1) serves correctly.
## Request
Restore the pure-fp8 1d1d kernels on `nv_dev` (or route fp8xfp8 on SM12x
back to a pure-fp8 kernel) so the fp8 path is correct again. vLLM's cmake
pin is currently `8b1392b`; we have opened a vLLM PR
([#53680](https://github.com/vllm-project/vllm/pull/53680)) to pin back to
`a6b593d` in the interim.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.