ROCm / ROCm/aiter

[Issue]: fused_qk_rmsnorm renamed to private _fused_qk_rmsnorm without deprecation (PR #2958)

Open
#3,207 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
565
Forks
585
Avg merge
3d 4h
Merged PRs (30d)
366

Description

Problem Description

Summary

PR #2958 ("add fused_qk_rmsnorm_per_token_quant kernel", merged May 9 2026) renamed the public function fused_qk_rmsnorm to the private _fused_qk_rmsnorm and changed its signature by prepending q_out and k_out parameters. This is a breaking change for downstream consumers.

Impact

vLLM's MLA dual RMS norm fusion pass (fuse_mla_dual_rms_norm) imports fused_qk_rmsnorm by name and calls it with the original 6-parameter signature. With AITER post-PR#2958, this produces an ImportError at runtime, breaking the fusion pass for all MLA models (DeepSeek-V3, Kimi-K2, etc.).

We have worked around this on the vLLM side (vllm-project/vllm#42606) by probing for both names with hasattr dispatch, but this approach requires maintaining two code paths indefinitely.

Request

  1. Provide a stable public name for the fused QK RMSNorm kernel. The current _fused_qk_rmsnorm (leading underscore) signals a private/internal API, which makes it risky for downstream projects to depend on. Either:

    • Re-export a public fused_qk_rmsnorm wrapper (can delegate to _fused_qk_rmsnorm internally), or
    • Document _fused_qk_rmsnorm as the intended public entry point and commit to its stability.
  2. Preserve backward-compatible signatures when possible, or use a deprecation period (e.g., keep the old name as an alias for one release cycle) so downstream consumers have time to adapt.

Context

Operating System

Ubuntu 22.04.5 LTS (Jammy Jellyfish)

CPU

AMD EPYC 9575F 64-Core Processor

GPU

AMD Instinct MI355X

ROCm Version

ROCm 7.2.2

ROCm Component

No response

Steps to Reproduce

No response

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

No response

Additional Information

No response

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing PR #2958 and the definitions and call sites of fused_qk_rmsnorm and _fused_qk_rmsnorm. Compare the old six-parameter usage in vLLM's fuse_mla_dual_rms_norm with the current signature, then verify that the chosen compatibility approach preserves downstream imports and calls without requiring indefinite dual dispatch.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai-infra-agents, backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.