Lightning-AI / Lightning-AI/lightning-thunder
Warn when more than N recompile is triggered.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
```python
import thunder
import torch
def foo(x, y):
return x + y
tfoo = thunder.jit(foo)
tcfoo = torch.compile(foo, dynamic=False)
x = torch.randn(10)
for i in range(10):
tfoo(x, i)
tcfoo(x, i)
print("Cache Misses:", thunder.cache_misses(tfoo))
```
Output
```python
W1021 10:44:18.989000 2926727 torch/_dynamo/convert_frame.py:1355] [0/8] torch._dynamo hit config.recompile_limit (8)
W1021 10:44:18.989000 2926727 torch/_dynamo/convert_frame.py:1355] [0/8] function: 'foo' (/opt/pytorch/lightning-thunder/test_thunder_recompile.py:4)
W1021 10:44:18.989000 2926727 torch/_dynamo/convert_frame.py:1355] [0/8] last reason: 0/7: y == 7 # return x + y # pt/pytorch/lightning-thunder/test_thunder_recompile.py:5 in foo
W1021 10:44:18.989000 2926727 torch/_dynamo/convert_frame.py:1355] [0/8] To log all recompilation reasons, use TORCH_LOGS="recompiles".
W1021 10:44:18.989000 2926727 torch/_dynamo/convert_frame.py:1355] [0/8] To diagnose recompilation issues, see https://pytorch.org/docs/main/torch.compiler_troubleshooting.html
Cache Misses: 10
```
It would be nice if thunder will also warn when there are a lot of recompiles.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the provided Python example with thunder.jit, torch.compile, and thunder.cache_misses to reproduce the ten cache misses and compare the output with PyTorch's recompilation warning. Trace how Thunder counts cache misses, then determine where a warning for exceeding a configurable N should be emitted; done means excessive recompilation produces a clear warning.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- compilers, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100