linkedin/Liger-Kernel

View on GitHub

Efficient Triton Kernels for LLM Training

Stars: 6.6k
Forks: 603
Open beginner issues: 0
Indexed issues: 110
Avg merge: 1d 20h
Merged PRs (30d): 47

Dominant language: Python
License: BSD-2-Clause
Last GitHub push: Sep 18, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: help wanted good first issue

Beginner friendly Everything

Open All

110 open issues indexed

JSD kernel returns NaN for interior beta when a vocabulary entry underflows under both distributions Open

Difficulty 3/5 1-2 days Newbie friendliness 76/100

linkedin/Liger-Kernel#1453 · Opened Sep 10, 2026
fused_linear_jsd rounds logits to the input dtype before the documented FP32 cast, costing up to 23% gradient error in bf16 Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

linkedin/Liger-Kernel#1432 · Opened Sep 1, 2026
[RFC] Migrate Nvidia CI from Modal to HF Jobs Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

linkedin/Liger-Kernel#1393 · Opened Aug 17, 2026
[RFC] Testing Methodology for FP32 Convergence Tests on Fused SwiGLU/dSwiGLU Kernels Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

linkedin/Liger-Kernel#1383 · 1 reaction · Opened Aug 15, 2026
Add Liger kernels + FLCE `token_accuracy` plumbing for Muse Glimmer (`muse_glimmer`) Open

linkedin/Liger-Kernel#1378 · 2 comments · 1 reaction · 1 assignee · Opened Aug 13, 2026
LigerGroupNorm backward writes out of bounds and then raises for any input with more than 3 dimensions Open

Difficulty 3/5 1-2 days Newbie friendliness 72/100

linkedin/Liger-Kernel#1375 · Opened Aug 11, 2026
LigerTVDLoss returns a non-zero gradient where p == q Open

Difficulty 2/5 1-3 hours Newbie friendliness 84/100

linkedin/Liger-Kernel#1373 · Opened Aug 11, 2026
[RFC] Full-pipeline SwiGLU MLP with forward/backward co-optimization Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

linkedin/Liger-Kernel#1347 · 6 comments · 1 reaction · Opened Aug 6, 2026
llama4_rope / qwen2vl_mrope kernels missing the int64 program_id cast that #804 added to rope.py/rms_norm.py Open

Difficulty 2/5 1-3 hours Newbie friendliness 70/100

linkedin/Liger-Kernel#1335 · 1 comment · Opened Aug 3, 2026
[B300/SM103] Compiled Liger SwiGLU is ~3.1x slower than torch.compile for a Qwen3-shaped BF16 tensor Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

linkedin/Liger-Kernel#1330 · 2 comments · Opened Aug 2, 2026
`torch.compile` fails with FLCE due to `aten.addmm.dtype_out` overload Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

linkedin/Liger-Kernel#1327 · Opened Jul 30, 2026
Reduce FLCE weight-gradient allocation overhead across FP16, BF16, and FP32 Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

linkedin/Liger-Kernel#1323 · Opened Jul 25, 2026
Support GMPO Chunked loss Open

Difficulty 5/5 Over a week Newbie friendliness 45/100

linkedin/Liger-Kernel#1315 · 1 comment · Opened Jul 23, 2026
Support Gemma 4 Unified (`gemma4_unified`) — kernel patching + fused linear cross-entropy Open

Difficulty 4/5 3-5 days Newbie friendliness 57/100

linkedin/Liger-Kernel#1308 · Opened Jul 21, 2026
Triton kernels can launch on the wrong GPU with multi-GPU inputs Open

Difficulty 3/5 1-2 days Newbie friendliness 65/100

linkedin/Liger-Kernel#1303 · 1 comment · 2 reactions · Opened Jul 16, 2026
pyproject.toml has no requires-python — pip installs on any interpreter; publish workflows disagree (3.8 vs 3.10) Open

Difficulty 2/5 1-3 hours Newbie friendliness 55/100

linkedin/Liger-Kernel#1302 · 1 comment · Opened Jul 15, 2026
Fused MoE kernels crash on Qwen3.5-397B (int32 offset overflow) Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

linkedin/Liger-Kernel#1296 · 1 comment · 2 reactions · Opened Jul 10, 2026
Add Qwen2.5-Omni support Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

linkedin/Liger-Kernel#1295 · 1 comment · Opened Jul 9, 2026
Feature Request: Add THD / Packed-Sequence Training Support Open

Difficulty 5/5 Over a week Newbie friendliness 28/100

linkedin/Liger-Kernel#1281 · 2 comments · Opened Jul 1, 2026
Suggestion: per-kernel roofline/bandwidth-efficiency benchmarks in docs Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

linkedin/Liger-Kernel#1272 · 1 comment · Opened Jun 27, 2026
LigerORPOTrainer doesn't work when not using FSDP Open

Difficulty 3/5 1-2 days Newbie friendliness 65/100

linkedin/Liger-Kernel#1229 · 1 comment · Opened May 19, 2026
Add fused Modulated RMSNorm for DiT-style scale/shift conditioning Open

Difficulty 5/5 Over a week Newbie friendliness 45/100

linkedin/Liger-Kernel#1224 · 5 comments · Opened May 13, 2026
[RFC] Add Monitoring Interface for External Logging Backends Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

linkedin/Liger-Kernel#1213 · 3 comments · Opened May 1, 2026
[New backend] cuTile backend for Liger kernels — performance data on B200 + integration interest Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

linkedin/Liger-Kernel#1205 · 1 comment · 4 reactions · Opened Apr 27, 2026
Gemma4 text model support Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

linkedin/Liger-Kernel#1186 · 5 comments · 5 reactions · Opened Apr 6, 2026
Glm4vConfig object has no attribute 'hidden_size' Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

linkedin/Liger-Kernel#1152 · 1 comment · Opened Mar 18, 2026
LigerFusedLinearGRPOLoss produces ~100x larger grad_norm than TRL's non-Liger-Kernel path due to missing vLLM IS correction and other differences Open

Difficulty 5/5 Over a week Newbie friendliness 42/100

linkedin/Liger-Kernel#1082 · 11 comments · 1 reaction · Opened Feb 7, 2026
Skip dW/dB computation in norm ops when weight/bias is frozen (LoRA/PEFT optimization) Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

linkedin/Liger-Kernel#1067 · 1 comment · 1 reaction · Opened Feb 4, 2026
[RFC] Standardized Model Config for Benchmarks Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

linkedin/Liger-Kernel#1051 · 6 comments · Opened Jan 29, 2026
Compatibility issue between liger-kernel and zero3: The size of tensor a (0) must match the size of tensor b (4096) at non-singleton dimension 1 Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

linkedin/Liger-Kernel#1048 · 1 comment · 1 reaction · Opened Jan 29, 2026
[NPU] Improve the performance of the embedding for NPU Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

linkedin/Liger-Kernel#1036 · Opened Jan 21, 2026
Fatal Python error when using Liger Kernel with Qwen-VL fine-tuning: `none_dealloc: deallocating None` at step ~875 Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

linkedin/Liger-Kernel#975 · 1 comment · Opened Dec 13, 2025
[NPU Roadmap, Updated to 2026-Q2] NPU support for Liger-Kernel Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

linkedin/Liger-Kernel#969 · 30 comments · 17 reactions · Opened Dec 8, 2025
Moe kernel for latest transformers v5 Open

linkedin/Liger-Kernel#958 · 4 comments · 5 reactions · 1 assignee · Opened Nov 28, 2025
[Feature Request] Add SwiGLU support for Qwen3-VL models Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

linkedin/Liger-Kernel#956 · 1 comment · 1 reaction · Opened Nov 28, 2025
[RFC] Native Ascend NPU Support for Liger Kernel Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

linkedin/Liger-Kernel#954 · 2 comments · 24 reactions · Opened Nov 26, 2025
Add GLM-4.5 MoE Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

linkedin/Liger-Kernel#951 · 1 comment · 1 reaction · Opened Nov 23, 2025
0.6.3 can not find unpack_cross_entropy_result in liger_kernel.transformers.model.loss_utils Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

linkedin/Liger-Kernel#944 · 1 comment · Opened Nov 18, 2025
[Question] No gain in VRAM usage with LigerFusedLinearCrossEntropyLoss Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

linkedin/Liger-Kernel#941 · 3 comments · Opened Nov 14, 2025
Another MLP implementation along with multiplier support Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

linkedin/Liger-Kernel#937 · 3 comments · Opened Nov 10, 2025
Investigate Helion for kernel authoring/tuning Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

linkedin/Liger-Kernel#923 · 3 comments · 1 reaction · Opened Oct 30, 2025
Check if cross entropy is applied correctly in all monkey patch functions Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

linkedin/Liger-Kernel#920 · 1 comment · Opened Oct 26, 2025
FusedLinearCrossEntropy got illegal memory access issue with triton 3.5.0 and PyTorch 2.9 Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

linkedin/Liger-Kernel#916 · 3 comments · Opened Oct 23, 2025
Support for Qwen3-VL models Open

high-priority

linkedin/Liger-Kernel#897 · 16 comments · 14 reactions · 1 assignee · Opened Oct 7, 2025
TiledMLP Open

help wanted

Difficulty 5/5 Over a week Newbie friendliness 30/100

linkedin/Liger-Kernel#893 · 13 comments · 1 reaction · Opened Sep 26, 2025
【FLCE】the bias in the previous layer Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

linkedin/Liger-Kernel#891 · Opened Sep 25, 2025
Support PyTorch Tensor Parallelism Open

linkedin/Liger-Kernel#868 · 1 assignee · Opened Aug 29, 2025
Multi-token attention Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

linkedin/Liger-Kernel#857 · Opened Aug 14, 2025
[QnA]: Why `cos` and `sin` is expected to be `hdim`, not `hdim//2`? Open

question

Difficulty 4/5 3-5 days Newbie friendliness 25/100

linkedin/Liger-Kernel#844 · 3 comments · Opened Aug 4, 2025
fused_linear_cross_entropy caused bad performance Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

linkedin/Liger-Kernel#827 · 4 reactions · Opened Jul 21, 2025

linkedin/Liger-Kernel

Get new issues in your inbox