linkedin / linkedin/Liger-Kernel

[NPU Roadmap, Updated to 2026-Q2] NPU support for Liger-Kernel

Open
#969 30 comments 17 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
6.6k
Forks
603
Avg merge
1d 20h
Merged PRs (30d)
47

Description

2026 Q2 Roadmap

Thanks to the Liger-Kernel team for their strong support of our work in Q1 2026. β€πŸ§‘πŸ’›πŸ’œπŸ€ŽπŸ’šπŸ’™

We will keep contributing to the Liger-Kernel + NPU community in Q2. This roadmap details the future plans for NPU native support. Welcome to join in the discussion.

Integrated into training software

In order to promote Liger-Kernel, we need to integrate it into real production scenarios to verify its performance in actual training and inference tasks.

We selected four training frameworks: VeOmni, LLaMA-Factory, verl and ms-swift, which are widely used in real production scenarios. We will continue to demonstrate the effectiveness of using Liger-Kernel in these software applications.

Software APIs PR
VeOmni LigerRMSNorm
liger_rotary_pos_emb
LigerSwiGLUMLP		 https://github.com/ByteDance-Seed/VeOmni/pull/415
LLaMA-Factory apply_liger_kernel_to_qwen3 https://github.com/hiyouga/LlamaFactory/issues/10386
https://github.com/hiyouga/LlamaFactory/pull/10493
verl _apply_liger_kernel_to_instance https://github.com/verl-project/verl/pull/6244
ms-swift apply_liger_kernel_to_qwen3_moe https://github.com/modelscope/ms-swift/pull/9227

Performance Enhancement: Part 2

Following this plan, the performance of some kernels still hasn't met the standard and will be continuously optimized in Q2. For example:

Skills

Due to the differences in programming logic between triton-ascend and triton, migrating kernels to NPU inevitably incurs a time cost for performance optimization.

To address this issue, we have developed a series of skills for AI programming tools.

We'll keep polishing these SKILL docs.

Support for more NPU machine types

Similarly, this includes three aspects: enabling functionality, ensuring accuracy within the error tolerance, and performance optimization.

Current support:

  • Atlas 900 A2 POD (64G), which can be used for benchmark tasks.
  • Atlas 800I A2 (32G), used by CI machines. This model NPU machine type not be used for benchmark work.

Planned support:

Improvement of NPU CI work

Currently, the NPU CI status badge can only be displayed in the README.

We plan to find a way to mark the CI status badge in NPU-related PRs. This would simulate the state after native CI integration and achieve the effect of checking PR quality.

2026 Q1 Roadmap

Thanks very much to Liger-Kernel for accepting our first native support pr.
This roadmap details the future plans for NPU native support. Welcome to join in the discussion.

NPU Native Support

This shows how Liger-Kernel works on NPU.

Unit Test Coverage Improvement: Functionality & Precision

The accuracy of all kernels in Liger-Kernel must be within the acceptable tolerance. This task can be accomplished by checking the actual execution results of all test cases in the ./test folder.

We have done the following work to ensure that all test cases under ./test/transformers have passed.

List of each kernel's first PR:

Additionally, the test cases under ./test/transformers can serve as the foundation for NPU CI to guard future pull requests.

Due to certain policy restrictions, Liger-Kernel cannot be natively integrated into NPU CI. Therefore, we have marked the NPU CI status badge in the README.

We will look for a more convenient solution in the future.

Performance Enhancement: Part 1

Unit tests can be used to APIs functionality and precision. However, as third-party devices may not fully align in their usage patterns of Triton, a performance optimization process is required.

Regarding the evaluation criteria, we have designed it as follows: The speedup ratio of each kernel compared to huggingface/torch should be greater than 1.

The progress of this task can be tracked in this rfc issue.

Benchmark Enhancement

To enable benchmarks to run reliably across devices with different memory capacities (e.g., 32G and 64G NPUs), we collaborated with the community to redesign the benchmark framework. The key improvements include:

  1. Device memory awareness: Introduced runtime memory probing (estimate_kernel_peak_memory) to automatically detect available device memory and determine safe execution parameters, ensuring benchmarks can run successfully on different device types without OOM.

  2. Standardized benchmark dimensions: Defined two orthogonal benchmark dimensions β€” D1 (non-model dimension sweep, e.g., sequence length) and D2 (model config sweep across real-world architectures from MODEL_REGISTRY), providing a more comprehensive and structured view of kernel performance.

Phase 1 (Foundation) and Phase 2 (Model-config sweep) have been completed. Phase 3 (Rollout and visualization) is planned for future work.

Co-Author: @momochen

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up β€” it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Begin with the Q2 roadmap and the linked performance plan in issue #1159; review the listed integration, benchmark, machine-support, and CI work. The issue does not identify a single file, test, or unclaimed task, so a contributor would need to choose and scope one item before coding. Done would require completing that selected item and validating it on the named NPU, test, or CI path.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, machine-learning, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.