Lightning-AI / Lightning-AI/lightning-thunder
Create a parametrized benchmark for LitGPT CausalSelfAttention
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
## 🚀 Feature
Create a parametrized benchmark for [LitGPT's `CausalSelfAttention`](https://github.com/Lightning-AI/litgpt/blob/d2626b0a6acbf93dd25199b04ad862bbe8ba8b9c/litgpt/model.py#L191).
Currently, we only test one input configuration for CSA for `Llama-2-7b-hf`
https://github.com/Lightning-AI/lightning-thunder/blob/7d62ae1c7da8fb57e09da3e95ced734e34480400/thunder/benchmarks/targets.py#L490
This benchmark should be parametrized similarly to `test_litgpt_qkv_split_rope`
https://github.com/Lightning-AI/lightning-thunder/blob/7d62ae1c7da8fb57e09da3e95ced734e34480400/thunder/benchmarks/targets.py#L572-L591
cc @crcrpar
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Open thunder/benchmarks/targets.py and inspect the existing CausalSelfAttention benchmark around line 490. Compare it with test_litgpt_qkv_split_rope around lines 572-591 to follow the repository's parametrization pattern. Done means the CausalSelfAttention benchmark covers parametrized input configurations rather than only the current Llama-2-7b-hf case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- performance, testing-qa
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100