linkedin / linkedin/Liger-Kernel

Suggestion: per-kernel roofline/bandwidth-efficiency benchmarks in docs

Open
#1,272 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
6.6k
Forks
603
Avg merge
1d 20h
Merged PRs (30d)
47

Description

Liger Kernel's README reports excellent aggregate numbers (20% throughput
increase, 60% memory reduction on LLaMA 3-8B). I haven't seen a per-kernel
breakdown of bandwidth efficiency (% of theoretical peak HBM bandwidth) for
the individual ops (RMSNorm, RoPE, SwiGLU, Softmax) — that level of detail
would help users understand *why* a given kernel is fast, and which ones
are closer to their hardware ceiling than others.

I built a small from-scratch reimplementation of these four kernels for my
own learning, with that exact kind of analysis (roofline-based, including a
documented failed optimization attempt on the RoPE kernel where "fixing" a
strided memory access pattern actually made performance worse):
https://github.com/ragulk143/inference-kernels

Not suggesting any change to Liger's actual kernels — they're clearly more
mature and production-tested than mine. Just wondering if a per-kernel
efficiency benchmark section (with a roofline-style table) might be a useful
addition to the docs for users trying to understand kernel-level performance
characteristics, not just aggregate end-to-end numbers.

Happy to contribute a draft if this is something the maintainers would find
valuable.

Image

Contributor guide

Open the contributing guide

Research direction

Start with the repository README and the linked roofline analysis for the RMSNorm, RoPE, SwiGLU, and Softmax kernels. Determine how per-kernel bandwidth-efficiency measurements can be documented, then add a roofline-style table and explain the documented RoPE optimization result; done means users can compare each kernel with the aggregate figures.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, machine-learning, performance
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.