Lightning-AI / Lightning-AI/lightning-thunder
Curious about the torch.compile baseline in benchmark_litgpt
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
Was curious to know why the [`reduce-overhead` option was not used as a baseline](https://github.com/Lightning-AI/lightning-thunder/blob/3bde9e695385ba37fb365c5ed1cd1cc71995aba2/thunder/benchmarks/benchmark_litgpt.py#L261) and if there's a comparison somewhere using `reduce-overhead` ?
Thanks !
cc @apaz-cli
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at thunder/benchmarks/benchmark_litgpt.py around line 261 and inspect how the torch.compile baseline is configured. Check whether a reduce-overhead comparison already exists; done would be a documented explanation of the baseline choice and, if absent, a reproducible comparison.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100