Lightning-AI / Lightning-AI/lightning-thunder
Create a parametrized benchmark for LitGPT layer norm
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
🚀 Feature
Currently, we only test one input configuration for layer_norm coming from nanoGPT:
https://github.com/Lightning-AI/lightning-thunder/blob/7d62ae1c7da8fb57e09da3e95ced734e34480400/thunder/benchmarks/targets.py#L239
This benchmark should be upgraded to use LitGPT's configurations parametrized similarly to test_litgpt_qkv_split_rope
https://github.com/Lightning-AI/lightning-thunder/blob/7d62ae1c7da8fb57e09da3e95ced734e34480400/thunder/benchmarks/targets.py#L572-L591
Similar request for RMSNorm: https://github.com/Lightning-AI/lightning-thunder/issues/741
cc @crcrpar
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in thunder/benchmarks/targets.py at the existing layer_norm benchmark around line 239, then compare it with test_litgpt_qkv_split_rope around lines 572-591. Adapt the layer_norm benchmark to use parametrized LitGPT configurations, and verify that the benchmark runs for each configuration; RMSNorm is tracked separately in issue #741.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 65/100