Lightning-AI / Lightning-AI/lightning-thunder

Warn when more than N recompile is triggered.

Open
#2,682 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.5k
Forks
121
PR merge metrics
No merged PRs in 30d

Description

```python
import thunder
import torch

def foo(x, y):
return x + y

tfoo = thunder.jit(foo)
tcfoo = torch.compile(foo, dynamic=False)

x = torch.randn(10)

for i in range(10):
tfoo(x, i)
tcfoo(x, i)

print("Cache Misses:", thunder.cache_misses(tfoo))

```

Output
```python
W1021 10:44:18.989000 2926727 torch/_dynamo/convert_frame.py:1355] [0/8] torch._dynamo hit config.recompile_limit (8)
W1021 10:44:18.989000 2926727 torch/_dynamo/convert_frame.py:1355] [0/8] function: 'foo' (/opt/pytorch/lightning-thunder/test_thunder_recompile.py:4)
W1021 10:44:18.989000 2926727 torch/_dynamo/convert_frame.py:1355] [0/8] last reason: 0/7: y == 7 # return x + y # pt/pytorch/lightning-thunder/test_thunder_recompile.py:5 in foo
W1021 10:44:18.989000 2926727 torch/_dynamo/convert_frame.py:1355] [0/8] To log all recompilation reasons, use TORCH_LOGS="recompiles".
W1021 10:44:18.989000 2926727 torch/_dynamo/convert_frame.py:1355] [0/8] To diagnose recompilation issues, see https://pytorch.org/docs/main/torch.compiler_troubleshooting.html

Cache Misses: 10
```

It would be nice if thunder will also warn when there are a lot of recompiles.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided Python example with thunder.jit, torch.compile, and thunder.cache_misses to reproduce the ten cache misses and compare the output with PyTorch's recompilation warning. Trace how Thunder counts cache misses, then determine where a warning for exceeding a configurable N should be emitted; done means excessive recompilation produces a clear warning.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
compilers, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.