linkedin / linkedin/Liger-Kernel
Suggestion: per-kernel roofline/bandwidth-efficiency benchmarks in docs
- Dominant language
- Python
- Stars
- 6.6k
- Forks
- 603
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 47
Description
Liger Kernel's README reports excellent aggregate numbers (20% throughput
increase, 60% memory reduction on LLaMA 3-8B). I haven't seen a per-kernel
breakdown of bandwidth efficiency (% of theoretical peak HBM bandwidth) for
the individual ops (RMSNorm, RoPE, SwiGLU, Softmax) — that level of detail
would help users understand *why* a given kernel is fast, and which ones
are closer to their hardware ceiling than others.
I built a small from-scratch reimplementation of these four kernels for my
own learning, with that exact kind of analysis (roofline-based, including a
documented failed optimization attempt on the RoPE kernel where "fixing" a
strided memory access pattern actually made performance worse):
https://github.com/ragulk143/inference-kernels
Not suggesting any change to Liger's actual kernels — they're clearly more
mature and production-tested than mine. Just wondering if a per-kernel
efficiency benchmark section (with a roofline-style table) might be a useful
addition to the docs for users trying to understand kernel-level performance
characteristics, not just aggregate end-to-end numbers.
Happy to contribute a draft if this is something the maintainers would find
valuable.
Contributor guide
Research direction
Start with the repository README and the linked roofline analysis for the RMSNorm, RoPE, SwiGLU, and Softmax kernels. Determine how per-kernel bandwidth-efficiency measurements can be documented, then add a roofline-style table and explain the documented RoPE optimization result; done means users can compare each kernel with the aggregate figures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation, machine-learning, performance
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100